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Executive Summary 


In the 2013-2014 school year, the Maryland State Department of Education (MSDE) launched the first 
year of full, state-wide impiementation of the Teacher and Principal Evaluation (TPE) system. The state 
TPE models consist of equally weighted qualitative measures on professional practices and quantitative 
measures on student growth. Local school systems may choose to implement the state TPE models, or 
develop their own plans that have measures aligned to local priorities, with the endorsement of their 
collective bargaining unit and approval from the state. 

MSDE support focuses on helping districts to prepare for full implementation. The state awarded more 
than $1 .5M in grants to districts last year in order to support software/hardware investments and teacher 
professional development on TPE. With ongoing input from districts, MSDE provides technical 
assistance with a focus on quality control, leadership development, and communications. In addition, 
districts can request customized technical assistance to support TPE implementation. 

This study examines frontline educators' perceptions of the TPE implementation in 2013-2014. It 
focuses on the support educators receive in understanding and implementing the TPE system and its 
components. In particular, this study examines the key TPE component of Student Learning Objectives 
(SLOs), an instructional process for improving educator effectiveness and student achievement. The 
Mid-Atlantic Comprehensive Center (MACC@WestEd), funded with a federal grant from the U.S. 
Department of Education, partnered with MSDE to conduct this study. The study team includes 
researchers from both Community Training and Assistance Center (CTAC) and WestEd. This study is a 
continuation of last year’s study of frontline educators’ experience with TPE in Maryland. Findings are 
intended to further inform and strengthen Maryland's implementation of the TPE system and the SLO 
component. 

Between mid-April and mid-May 2014, researchers collected both qualitative and quantitative data from 
educators across the state, including (a) interviews of superintendents, teacher association leaders, 
central administrators, and principals from 17 districts; (b) focus groups of teachers from four districts; 
and (c) a statewide survey of teachers, principals and other educators. The final survey analysis is based 
on the responses of 16,314 educators from 23 local school districts, with district response rates ranging 
from 5.7% to 72.6% and a collective 23 district response rate of 31 .3%. 

Findings 

On many issues, the perceptions of Maryland’s frontline educators are positive about the potential of 
TPE. As they have more experience in implementing the new evaluation system, teachers and principals 
indicate that their confidence and skill levels are growing. They also raise concerns that need to be 
addressed if the quality of implementation is to increase. 

Overall Perceptions of TPE 

• When implemented thoughtfully and purposefully, educators say that TPE is prompting deeper 
analysis and use of data to focus on student needs. 

• Principals are more likely than teachers to agree with statements about the positive implications 
of TPE. 
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Effect of Prior TPE Experience 

• Districts or schoois that had prior experience with TPE are impiementing more effectiveiy, with 
iess stress, and more people view the reform positively. 

TPE impiementation with Different Teacher Popuiations 

• Veteran teachers (1 0+ years) are more likely to agree than disagree that the implications of TPE 
are positive. Teachers who are relatively new to teaching (0 to 3 years) are even more positive 
than veteran teachers about TPE and its potential impacts. 

• Elementary teachers are more likely to agree about the positive implications of the TPE system 
than are teachers in middle school or in high school. 

Schooi, District and State Support 

• Educators indicate that the key support they receive is from their own district or school. 

• Superintendents credit MSDE with effective help with TPE implementation. 

instructionai Diaiogue 

• Almost three-quarters of principals agree that reflection on instructional practices has deepened 
and there is more instructionally focused dialogue. Teachers are as likely to disagree as agree 
that reflection deepened or that there is more instructional dialogue. 

Teacher Ratings 

• Many educators say that overall teacher ratings are not going to change very much from the 
previous evaluation system. 

• In districts that view the TPE process as part of the instructional system, teachers have few 
concerns about ratings, but in districts that view the TPE process more as compliance with 
mandates, teachers are concerned about receiving poor ratings, or losing their jobs. 

Common issues in impiementing TPE 

• Interviewees across role groups suggest that the pace of implementation has been unrealistic, 
limiting the development of knowledge and classroom support systems. 

• Districts are struggling to make the connections between SLOs, observations, and the new 
Maryland College and Career-Ready Standards. 

• TPE implementation is inconsistent across classrooms, departments and schools. Gaps in 
communication make it more difficult for teachers to contribute to efforts to strengthen TPE. 

• Many educators are concerned about the skills and capacity of principals, wanting more training 
in instructional leadership, conducting observations, and managing time demands. 

• Teachers say they need more training on the components of TPE, using standards, and using 
data. This need is more acute in districts with a high percentage of students receiving free and 
reduced price lunches. 

• Districts are struggling with the technology needed to manage TPE-related data. 

impiementing Teacher Observations 

• Both teachers and supervisors are positive about using observations in the TPE process, but 
they are still building their understanding of the instruments. 

• 44% of teachers agree that TPE utilizes validated observation measures/instruments. 

• 61 % of teachers agree that TPE observations are conducted by qualified observers. 

• Teachers indicate that the observation process varies widely between and within schools. 
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Implementing SZ.Os/ The interviews and survey responses show the openness of Maryland educators to 
using SLOs to advance student achievement and professional practice— but they want to be supported 
in doing so. 

Overall Expectations for SLOs 

• Educators like the idea of using SLOs as a tool to measure student growth and facilitate 
collaborative discussions on Instructional strategies and teaching effectiveness. 

• Districts and schools with prior TPE training or Implementation experience are more confident in 
implementing all aspects of SLOs this year. 

Experience with 201 3-201 4 Implementation 

• Principals have a more positive Impression than teachers that SLOs are being implemented 
effectively, that data are used to develop SLOs, that there are opportunities to confer with 
principals about SLOs, that there are opportunities to engage in a mid-year review, and that 
evaluation results will be used to Inform professional development plans. 

• 30% of teachers do not know whether the results of TPE evaluation will be used to Inform 
professional development plans, and 25% expect that they will not. 

• 43% of teachers agree that they are receiving professional development that is informed by the 
SLO process, 37% do not agree. 

• Many teachers indicate they want more input into how SLOs are selected and crafted. 

Challenges of Implementing SLOs: Interviewees like the promise of SLOs, but many are finding the 
implementation of SLOs in their school or district to be challenging for a variety of reasons. 

• Interviewees have concerns about principal readiness to effectively guide the SLO process. 

• Many Interviewees believe that teachers receive insufficient training on how to understand and 
develop an SLO. Teacher and principal perceptions differ on whether teachers receive enough 
support on: guidance to select Instructional strategies. Information about developing high quality 
SLOs, valid and reliable approaches to setting growth targets, and recommendations for 
selecting pre/post assessments. 

• Most Interviewees grapple with understanding what constitutes a high quality SLO. 

o Many educators would like a quality rating rubric to help guide the development and 
review of SLOs. Very few Interviewees report having such a rubric, 
o Interviewees want to learn more about how to analyze the strengths and weaknesses of 
specific SLOs. 

Recommendations 

Being able to successfully manage the TPE system Is a core requirement for every district in Maryland. 

In making decisions to advance TPE, implementation needs to be sensitive to important dimensions of 
educator practice. This means that districts must strike a thoughtful balance between manageability and 
having a thorough, rigorous process, and between standardization and customization. In making 
decisions on this front, districts would benefit from state-provided guidance on how much flexibility 
schools should be allowed to adapt and contextualize processes, while still having the comparability to 
maintain confidence and trust in the overall TPE system. 
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This balance is essential when implementing SLOs. The SLO process should foster the planning and 
delivery of instruction through the use of data and systematic analysis. Further, it should motivate 
performance though the setting of growth targets by both teachers and students. Districts wouid 
markediy benefit by heeding a critical national lesson learned from fifteen years of SLO practice and 
research: The process of thinking through the inter-related components of the SLO and their 
impiications for instruction is what matters most. 

The following recommendations, detailed in chapter six, focus on what the state can directly support, 
and what district and schooi ieaders need to do weii, in order to have a high quality implementation of 
the TPE system in general and of the SLO process in particular. They focus on ways to improve the 
quality and consistency of impiementation within and across districts in Maryiand, whiie reinforcing the 
instructionai emphasis of TPE. 

The recommendations build on the comprehensive support MSDE is currently providing to LEAs. They 
reinforce the Influencing Transformation strategy that guides MSDE’s support to districts. These 
recommendations also should become part of the agenda of the alliance of eight organizations that have 
come together in Maryland to support the effective implementation of SLOs as part of the overall TPE 
system. 

MSDE has established a strong foundation for implementing TPE. The next phase of implementation 
improvements needs to come at district-to-school and school leader-to-teacher levels. While MSDE will 
continue to provide a range of supports to LEAs, ownership of continuous improvement of TPE needs to 
increase at the local level. 

Issue 1: TPE Integration with Instruction 

• Provide a series of instructional crosswalk resources to districts. 

• Deveiop protocois to ieverage evidence that can inform professionai deveiopment and 
ieadership deveiopment. 

• Buiid district capacity to assess and instructionaiiy integrate their TPE impiementation. 
issue 2: Principal Leadership 

• Deveiop an expanded series of principal leadership prompts. 

• Provide a specific stream of support customized to high schooi principais and their immediate 
supervisors. 

• Convene a statewide working group of principals. 

Issue 3: Quality and Consistency of SLO Implementation 

• Provide districts with a sampie SLO Quaiity Rating Rubric. 

• Broaden capacity buiiding to address emerging need areas. 

• Provide a resource iibrary of annotated SLOs. 

• Convene district ieaders to examine the requirements of an instructionai management system 
for their impiementation of TPE. 

Issue 4: Quality and Consistency of Observations 

• Provide further resource materials on key aspects of the teacher observation process. 

• Provide additionai professionai deveiopment toois to heip teachers and principais deveiop better 
understanding of the observation rubrics and the teaching components. 
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Issue 5: TPE Data Management and Technology Infrastructure 

• Convene district ieaders to identify the criteria for and capabiiities of technoiogy soiutions to 
help manage the TPE process. 

Issue 6: Constituency Building and Inter-Dlstrict Learning 

• Convene a working group on two-way communications. 

• Develop an interactive web site for sharing vetted practices. 

Summary 

MSDE is making significant strides in guiding and supporting the implementation of SLOs as well as TPE 
overall. The more experience frontline educators have with the new evaluation system, the higher their 
skill and comfort levels are with its implementation and the more their efforts focus on strengthening 
instruction. The dual challenge of improving the quality and consistency of implementation within and 
across districts is not unusual in a major new state undertaking. With the ongoing high quality support 
from the state, districts can directly address this challenge through thoughtful and coordinated action in 
six key issue areas. Maryland’s landmark alliance of eight pivotal organizations further strengthens the 
foundation and prospects for continued progress in the year ahead. 
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Chapter One: Introduction 

Purpose of This Study 

The 2013-2014 school year marked the first year of full implementation of the statewide Teacher and 
Principal Evaluation (TPE) system, and the Maryland State Department of Education (MSDE) is interested 
in learning from and with educators and refining the implementation of the initiative. To help the state in 
this area, the Mid-Atlantic Comprehensive Center (MACC@WestEd), funded under a federal grant from 
the U.S. Department of Education, partnered with MSDE and conducted this study. The study team 
includes researchers from both Community Training and Assistance Center (CTAC) and WestEd. The 
following study is a continuation of last year’s successful joint endeavor to learn first-hand from frontline 
educators in Maryland. 

This study examines the perceptions of frontline educators in Maryland of the support they receive in 
understanding and implementing the new TPE system. It particularly focuses on the key component of 
Student Learning Objectives (SLOs), an instructional process for improving educator effectiveness and 
student achievement. It also explores educators’ perceptions of TPE implementation in 2013-2014 and 
additional supports they need. Findings of this study are intended to help further inform and strengthen 
the implementation of the overall TPE system and the SLO component in Maryland. 

Background of This Study 

Maryland is implementing a statewide TPE system, which includes both professional practice measures 
and student growth measures. The evaluation results are intended to inform and engineer targeted, 
supportive, and timely professional development and strengthen the knowledge, skills, and teaching 
practices of educators to improve student learning. 

During the 201 1 -201 2 school year, a pilot of the TPE system was implemented in seven school districts: 
Baltimore City, Baltimore County, Charles County, Kent County, Prince George’s County, Queen Anne’s 
County, and St. Mary’s County. The pilot consisted of 83 schools, 934 teachers, and 48 principals. 

In 2012-2013, a field test was conducted in all schools within 22 school districts, all of which signed on 
to implement the Race to the Top grant. Three school districts (i.e., Anne Arundel County, Calvert 
County, and Somerset County) implemented the full state model and the other 19 districts developed 
their own local models approved by the state. 

As noted above, full statewide implementation of TPE took place in 2013-2014. 

Maryland's TPE System 

The state TPE models consist of both qualitative measures on professional practices and quantitative 
measures on student growth. Both are equally weighted. 

The professional practice for teachers consists of four domains outlined by the Charlotte Danielson 
Framework for Teaching (i.e., planning and preparation, instructional practices, classroom environment, 
and professional responsibilities) and the four domains are further broken down into 22 components. 
Each of the four domains is worth 12.5 percent of the professional practice section, totaling 50 percent 
of the total evaluation score. For principals, the professional practice measure comprises of eight 
domains based on the Maryland Instructional Leadership Framework and four domains from the 
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Interstate School Leaders and Licensure Consortium. These 12 domains are weighted individually to 
reflect the differential needs of principals at varying times in their careers. 

Student growth and achievement is measured through the School Performance Index (SPI) and SLOs. 
The SPI consists of school strands ranking schools from 1 (lowest) to 5 (highest). SLOs are the 
predominate measure of student growth for teachers and principals. The state TPE models allot for 30% 
-50% of the total evaluation rating to SLOs, depending on the assignment of the teacher and principal. 
No single SLO, however, may count for more than 35 percent of the total performance score (see 
Figures 1 and 2 in the addendum for the state’s graphics which illustrate these models). 

District Variations 

In addition to adopting the state modeis, iocai schooi systems may choose to deveiop iocai pians and 
include additional measures that align to their local priorities. These local plans are required to include 
certain criteria and measures defined by the TPE framework that every evaluation model must have (e.g., 
the 50/50 split between professional practices and student growth, and the four Danielson-like domains 
for teachers and the eight Maryland Instructional Leadership Framework domains for principals). 
Meanwhile, local models need to have the endorsement of the local collective bargaining unit and must 
receive approval from the state. 

The variance among district models is small. Most districts follow the state model for professional 
practice and the few different models cross walk to the state. All districts embrace SLOs, but the 
number and weighting of SLOs vary. 

Because the historical and current practice in Maryland is one of local control, the key implementation 
decisions are made and the organizational supports that reinforce those decisions are provided at the 
district level. Accordingly, the findings presented in this report related to TPE implementation are based 
on educators’ perceptions of practices within their respective districts and schools. 

MSDE Ongoing Support for TPE Implementation 

In fall 2012, MSDE formed a TPE Action Team to serve LEAs. The team had four strategic sub-teams: 
communications, field test. Student Learning Objectives (SLOs), and professional development (other 
than SLOs). The TPE Action Team began publishing bi-weekly communication newsletters that outline 
plans and progress of each of the sub-teams, and cover all aspects of TPE implementation and MSDE 
responses to concerns raised by districts. The twenty-sixth bi-weekly communication came out in July 
2014. 

In May 2013, the TPE Action Team re-configured to respond to changing needs of districts in 
implementing TPE. The new sub-teams focused on evaluation, communications. Student Learning 
Objectives (SLOs), and leadership development. The redesign included a particular focus on readiness, 
to support districts in preparing for full implementation of TPE. 

LEAs did not want a centralized state system. To assist LEAs in developing their customized TPE 
systems, MSDE released over $1 .5M in mini-grants to LEAs. Grants were awarded by formula which 
included inverse size, wealth, burden, and academic challenge. The largest percentage of grants to LEAs 
were for school hardware/software investments (31.4% of grants), followed by district 
hardware/software investments (18.1 %), teacher professional development on TPE, the Danielson 
framework, and SLOs (8.9%), and inter-rater reliability (7.0%). 
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The SLO team created an SLO sub-group with district representatives, deveioped an SLO professionai 
development pian, and provided extensive SLO training and online resources statewide. 

The communications team provided support to educators statewide through its MSDE Teacher and 
Principal Evaluation Website (http://marviandpubiicschoois.org/MSDE/proarams/tpe/index.htmi) . The 
website has all the documents and communication materials produced by the state related to TPE 
implementation. This site includes The Maryland Teacher and Principal Evaluation Guidebook, Version 3 
(http://msde.state.md.us/tpe/TPE Guidance Versions 09201 3.pdf) . as well as the twenty-six bi-weekly 
communication newsletters that cover all aspects of TPE implementation and MSDE responses to 
concerns raised by districts. 

The leadership development team created a leadership development sub-group with district 
representatives, coordinated all non-SLO professional development activities, assisted in the 
coordination of professional development efforts across Maryland’s College and Career-Ready 
Standards, TPE, and PARCC, and delivered professional development. The team developed a TPE 
professional development plan to ensure that districts receive training prior to each phase of TPE 
implementation. The team released a diagram of the cycle of professional development that should 
precede each stage of TPE evaluation (see Figure 1). 


Figure 1: Building the TPE Professional Development Calendar 


Translating 
MSA to a % 


Extracting Teacher 
& Principal Measures 


Setting SlOs 

Coftducting a 
pre-conference 





Source: Teacher and Principal Evaluation (TPE) Communication #16, May 16, 2013, 
http://msde. state. md.us/toe/Communication 16.pdf . 
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By fall 2013, with input from districts, the TPE Action Team turned its attention to fuli implementation of 
the TPE models, with a focus on quality control, technical assistance, leadership development, and 
communications. Since districts are responsible for their own implementation of TPE, the TPE Action 
Team focused on “Infiuencing Transformation” as a way of sustaining the reform. The team split the 
implementation work into five Sphere of Infiuence cycles to manage the work and respond to district 
needs. 

The foci of each Sphere are as foliows: 

• Sphere 1 (July 1-Sept 19): developing SLOs, selecting MSAs, and conducting pre-conferences 

• Sphere 2 (Sept 20-Oct 31): visiting schools to observe professional practice, observing 

professional practice, and connecting Common Core State Standards to observation 

• Sphere 3 (Nov 1-Jan 31): monitoring SLOs, and conducting a mid-year SLO check 

• Sphere 4 (Feb 1-May 1): attributing students to teachers, scoring professional practice, scoring 

SLOs, and setting professional practice goals 

• Sphere 5 (May 2-June 30): completing an effectiveness rating, reviewing annual data and aligning 

school improvement to evaluation 

Within each Sphere cycle, the TPE Action Team held meetings with LEA professional development 
coordinators, executive officers, PSSAM executive board, assistant principals, MASSP, assistant 
superintendents, and superintendents. These meetings provided professional development to key LEA 
personnel responsible for TPE activities in the districts. 

The Action Team published bi-weekly communication bulletins focused on information sharing, technical 
content, quality controls, and accomplishment of Sphere goals. The team also held quality control 
meetings to gauge implementation milestones and issues and to determine district readiness for the 
next Sphere. Furthermore, the TPE Action Team met with LEAs by request to provide additional 
localized training on the current topics. 

Finally, the TPE Action Team maintained data on LEAs' self-reported readiness for each aspect of TPE 
implementation. This allowed the TPE Action Team to re-visit topics where districts indicated they were 
not ready for implementation. Polling LEAs on issues also gave the TPE Action Team the ability to 
respond quickly to emerging issues. 

Teacher and Principal Guidance on Implementing SLOs 

In the Maryland Teacher and Principal Evaluation Guidebook, Version 3, September 2013, MSDE 
emphasizes using SLOs as an ongoing, iterative and collaborative process focusing on data review, 
rigor, collaboration, refining instruction, and professional growth. It outlines the following steps for SLO 
implementation: (a) professional development; (b) data review; (c) SLO development; (d) review and 
approval conference; (e) mid-interval conference; (f) final SLO review; (g) integration of SLO results; (h) 
planning next steps; and (i) setting the attribution roster. 

To support teachers and principals in crafting high quality SLOs, MSDE developed the following tools: 

(a) SLO template for teachers; (b) guiding questions for teachers to write SLO; (c) SLO tempiate for 
principals; and (d) guiding questions for principals to write SLO. In addition, MSDE developed guidance 
materials which provide criteria in four critical domains to assist with the review and approval process: 

(a) priority of standard; (b) rigor of target; (c) quality of measure and evidence; and (d) action plan. Also, 
MSDE encourages the development and use of team SLOs for teachers and common SLOs for 
principals. 
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It is the purview of local districts to estabiish processes based on guidance from MSDE for setting, 
reviewing, assessing, and aligning SLOs to school improvement pians and to LEA, State, and Federal 
priorities. In addition, districts are expected to provide SLO training to iocai schooi personnel in keeping 
with the established state guidelines. They are also responsible for developing and documenting a 
verification process to vaiidate the consistency, comparability, quality and rigor of SLOs and the 
evaluation results. 

Teacher and Principal Guidance on Implementing Classroom Observations 

MSDE provides guidance to support the impiementation of the classroom observation component in 
TPE. Some key instructions on evaluating educators’ professional practice are listed below. 

• Classroom observations shall be conducted by certificated individuals who have completed 
training that includes identification of teaching behaviors that result in student growth. 

• An evaluation of a teacher’s professional practice shall be based on at least two observations 
during the school year. 

• Non-tenured and ineffective teachers shall be evaluated annually on professional practice and 
on student growth measures. 

• Tenured and satisfactory or effective/highly effective teachers shall be evaluated on a three-year 
cycle using a phase-in model. Each district is required to determine a methodology for schools 
to initially identify proportional balancing of their tenured teachers for the continuous rolling 
evaluation plan. 

• An observation, announced or unannounced, shall be conducted with full knowledge of the 
teacher. An evaluation report that evaluates a teacher as ineffective shall include at least one 
observation by an individual other than the immediate supervisor. 

• A written observation report shall be shared with the teacher and a copy provided to the teacher 
within a reasonable period of time. A certificated individual shall sign the observation report to 
acknowledge receipt; and 

• An observation shall provide for written comments and reactions by the teacher being observed, 
which shall be attached to the observation report. An observation shall provide specific 
guidance in areas needing improvement and supports as well as a reasonable timeline to 
demonstrate improvement in areas marked as ineffective. 

Layout of the Report 

The remainder of this report is organized as follows. Chapter two describes the methods of data 
collection. Chapters three and four examine the overall TPE implementation and the common issues. 
Chapter five focuses on the implementation of SLOs, a major component of and vehicle in TPE to 
improve educator effectiveness and student learning. Finally, chapter six summarizes the key issues and 
presents recommendations. 
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Chapter Two: Methods of Data Collection 


Both qualitative data and quantitative data were collected from educators across the state, including (a) 
interviews of superintendents, teacher association leaders, central administrators, and principals from 17 
districts; (b) focus groups of teachers from four select districts; and (c) statewide survey responses 
provided by teachers, principals and other educators from 23 districts. 

Key Leader Interviews 

Confidential phone interviews were conducted from late April to early May with 25 key stakeholders in 
17 districts and one statewide association. These include: nine superintendents, six local teacher 
association leaders, one statewide teachers’ association leader, two central administrators, and seven 
school principals. Using interview protocols developed by the study team, each interview lasted for 
approximately one hour. 

Teacher Focus Groups 

In early May, four confidential teacher focus groups were conducted with 39 frontline teachers in four 
districts. Participants of the focus groups were identified by the districts and composed of ten teachers 
on average representing a cross-section of elementary, middle, and high schools teachers, and tested 
and non-tested area teachers. Using interview protocols developed by the study team, each focus group 
lasted for approximately one hour and a half. 

Maryland Teacher and Principal Evaluation Survey 

A web-based confidential survey, developed by the study team, was launched on April 21, 2014. The 
Maryland State Department of Education (MSDE) sent an invitational email with the link to the online 
survey to all superintendents in the state, asking them to share this information with the principals, 
teachers, and other school-based staff in the local school districts and encourage them to respond. The 
study team alone had access to survey responses. MSDE updated the superintendents regularly on the 
progress of survey participation. The survey was open four weeks until May 16, 2014. 

The 39 survey questions centered on: (a) the systemic context for the implementation of the TPE system; 
(b) the quality of the TPE frameworks and processes; (c) supports teachers receive on Student Learning 
Objectives (SLOs); (d) educators’ experience with the TPE system during the 2013-14 school year; and 
(e) additional supports educators need. 

Originally, 16,407 educators from all of the 24 districts responded to the survey. A total of 93 surveys 
were excluded from the study for the following reasons: 86 participants did not identify their district 
affiliations, and seven participants were from the district in which fewer than 10 educators responded to 
the survey. The final analysis was based on the responses of 16,314 educators from 23 local school 
districts. The district response rates ranged from 5.7% to 72.6%, with a state average of 31 .3%. 

A substantial number of educators have gained knowledge about and experience with SLOs and the 
new TPE system. The majority of the respondents have participated in SLO trainings (86%). Over two in 
five respondents have participated in classroom observation trainings (44%), and nearly half have 
accessed some sources of information or experiences related to SLOs (49%). 


Real Progress in Maryland: Student Learning Objectives and Teacher and Principal Evaluation 


11 


The majority of the respondents are classroom teachers (68%), and nearly two-thirds of the teachers 
have 10 or more years of teaching experience (63%). Most of the survey respondents have a Master’s 
degree (76%). A total of 622 principals completed the survey, constituting 42.9% of all school principals 
in the state. (See Table 2 in the addendum for more detail on the characteristics of the survey 
respondents.) 

The survey analysis includes an examination of the responses by three teacher categories: classroom 
teacher, non-classroom teaching staff, and special educator. Since teachers’ responses are very similar 
across their categories, they have been grouped into one category, “Teacher,” for purposes of 
discussion in the report. For simiiar reasons, the principai and assistant principai responses have been 
grouped into one category, “Principai.” 

Limitations of the Data 

In order to gain actionable information and timeiy insights in the impiementation of the TPE system, data 
were collected and studied within a relatively short time frame. Although educators’ responses from 23 
of the 24 districts in the state were analyzed in the survey, the response rates by district varied 
considerabiy. In addition, the data were perceptual in nature and no performance data (e.g., teacher 
ratings, student test scores) were able to be gathered or examined at the time of this study. 

In spite of these limitations, however, consistent findings emerge from the interviews, focus groups, and 
survey responses, drawing a clear picture of the strengths and challenges of TPE and SLOs. These 
findings have important policy and leadership implications for the continued implementation and 
improvement of the system down the road. 
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Chapter Three: TPE Implementation in Maryland 


The analysis of findings begins in this chapter by examining themes that emerge from interviews and 
surveys about the TPE system and its implementation. This section covers: educators’ overall 
perceptions of TPE; the effect of prior experience with either SLOs or observations; how different 
teacher populations view TPE; what source of support educators value; how TPE affects instructional 
dialogue; and how educators view teacher ratings. 

Overall Perceptions of TPE 

As one of the first states to receive Race to the Top funding, Maryland is in a lead role nationally in 
working to improve its teacher and principal evaluation system. Maryland’s statewide approach is rooted 
in instructional improvement, with an emphasis on helping teachers better analyze and use student data 
to individualize classroom instruction. Because Maryland is a local control state, the Maryland State 
Department of Education (MSDE) is advancing this approach by collaborating with local educators to 
develop new understandings, strengthen educator practices and build local constituencies supportive of 
the reform. 

In interviews, educators across role groups have a largely positive perception of TPE’s potential. They 
report that when implemented thoughtfully and purposefully, TPE is prompting deeper analysis and 
use of data to focus on student needs. In these schools and districts, this focus on student needs is 
helping teachers to tailor instructional practices in order to meet student needs. Many teacher 
interviewees are hopeful about the promise of the TPE system to improve practice, but they are even 
more sanguine if they work in a district where they feel that their voices are being heard on decisions 
related to TPE. That involvement gives them more confidence about how the various pieces of TPE fit 
together and why a teacher would receive a particular rating. 


"TPE is getting teachers to look more closely at the data and how the data are impacting 
instruction. In the past, there were more generalizations. Now we are looking at how we are 
going to change what we are doing in the classroom." 

-Teacher 

"TPE is very data driven. This is huge. Before this, it was, 'Sorry, buddy, you didn't make it.' 
Now you have to look at those kids and provide focused instruction. It makes differentiation a 
necessity. " 

-Teacher 

"It makes teachers think. There does need to be some accountability. Although it is hard, it is a 
good thing to do. " 

-Teacher 


It is important to note, that in the statewide survey of Maryland educators, principals have positive 
opinions about TPE and its expected outcomes. Principals have the primary role in implementation in 
their building and first hand awareness of how implementation is going in their buildings. Principal 
responses show high levels of agreement that the expectations for TPE are clear (70%), that TPE is 
based on principles of continuous improvement (86%), and that TPE recognizes the scope of an 
educator's roles and responsibilities (71 %). 
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On these same items, teachers’ levels of agreement are lower than the principals, but more teachers 
agree than disagree that expectations for TPE are clear (50%), TPE is based on principles of continuous 
improvement (60%), and TPE recognizes the scope of an educator's roles and responsibilities (46%). 


Figure 2: Principal and Teacher Views of the Context for TPE 
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Teachers 


The expectations are clear for 
teachers and principals in the 
TPE system.* 


Principals 


Teachers 


TPE is based on principles of 
continuous improvement.* 


Principals 


Teachers 


TPE recognizes the scope of 
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responsibilities.* 


I Disagree 
I Undecided 
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Note: * indicates statistically significant differences at the 95% confidence level. Agree \s a composite of strongly 
agree/agree. Disagree \s a composite of strongly disagree/disagree. 


As indicated in chapter one, districts have the 
option of adopting the state models for TPE or 
adapting the state model to best fit contextual 
needs. Inherent in making district level decisions 
about new evaluation systems, interviewees talk 
about the dynamic of balancing the rigor of the 
system with realistic goals, as well as balancing 
the need to standardize the process while being 
flexible with the model. In many of the districts, 
educators indicate that they were making 
adjustments throughout the year to make the 
model work in their local context. 


"I thought I had set a lofty goal. When 
1 examined the data, 1 saw that the 
goal was not as lofty as 1 had 
thought. So I made a correction and 
set it higher. It was reset to be an 
attainable but more challenging 
goal. " 

-Teacher 




"Whether people are happy or not with TPE, there is a degree of consensus that the model is 
realistic. We are getting better with SLOs and they will align to what we are saying about 
performance . " 

-Superintendent 


"We look at SLOs carefully. We are not in 'one size fits all.’ We are into customization." 

-Superintendent 
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Effect of Prior TPE Experience 


Districts or schools that had prior experience with TPE components are implementing more 
effectively, whether that experience is from receiving substantive training, participating in the pilot 
test, or participating in serious field testing. Teachers in these districts have more knowledge about 
TPE and are more comfortable with its components. With advance preparation and focused support, 
these schools and districts have a leg up on those that did less training or piloting or approached the 
work more from a compiiance perspective. Teachers and principals in schools that piloted or field tested 
TPE in a comprehensive manner say school staff feel more confident in what they are doing and have 
less fear about the process. 


"If you asked me this question last year, I would say, 'Lord, this is so much.' We now are more 
comfortable with the tools. It's becoming more manageable." 

-Principal 

"My school did well. We had the pilot and 1 had the training. 1 can't imagine those schools that 
just started." 

-Teacher 

"We were a pilot county last year. . .We had regular meetings, and we tweaked the whole 
evaluation process. That was a collaborative effort." 

-Principal 


Survey data confirm that educators who participated in trainings on classroom observation and/or SLOs 
are more likely to agree on almost every question on the survey (such as the expectations are clear, TPE 
is based on principies of continuous improvement, TPE recognizes the scope of an educator’s roles and 
responsibilities) than are educators who neither participated in trainings nor accessed information (see 
Tabie 1). 

Of particuiar note is the ievei of comfort with SLOs. Over one-haif of teachers and principals who 
received training indicate that teachers receive information about developing high quality SLOs (54%), 
the use of SLO components (54%), guidance on selecting learning content (54%), and the use of pre- 
and post-assessments for SLOs (51 %), compared to teachers and principals who neither participated in 
trainings nor accessed TPE information. Simpiy put, training makes a difference for participants in TPE. 

Table 1: Responses of Teachers and Principals with and without TPE Training 
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Note: * indicates statistically significant differences at the 95% confidence level. Agree \s a composite of strongly 
agree/agree. Disagree \s a composite of strongly disagree/disagree. 

Educators report that the stresses of implementation during the first year lessen in the second 
year, and people begin to view the reform more positively. Survey results suggest (see Table 2) that 
in the second year of implementation, more teachers and principals are more positive about many 
aspects of the TPE system. Some of the strongest increases relate to SLOs. In the table that follows, the 
change in the percentage of principals and teachers, respectively, who agree with the statements is 
indicated in yellow and red. 
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Table 2: Year to Year Change in Educator Survey Responses Who Agree with the 
Statements from 2013 and 2014 TPE Surveys for All Respondents 



Principals 

Teachers 

Systemic Context 

2013 

2014 

Change 

2013 

2014 

Change 

The expectations are clear for teachers and principals in the 
TPE system. 

55% 

70% 

+15%* 

41% 

50% 

+9%* 

TPE is based on principles of continuous improvement. 

83% 

86% 

+3% 

58% 

60% 

+2% 

TPE recognizes the scope of an educator’s roles and 
responsibilities. 

67% 

71% 

+4% 

42% 

46% 

+4% 

Quality of TPE Frameworks and Processes 

Teacher evaluation frameworks and processes... 

Respect educators’ professional knowledge and skills. 

82% 

85% 

+3% 

51% 

53% 

+2% 

Utilize validated observation measures/instruments. 

67% 

76% 

+9% 

40% 

44% 

+4% 

Provide a summative rating of educator performance. 

71% 

84% 

+13%* 

42% 

51% 

+9%* 

Lead to improved decisions related to instructional 
approaches. 

66% 

79% 

+13%* 

38% 

48% 

+1 0%* 

Student Learning Objectives (SLOs) 

Teachers receive... 

Information about developing high quality SLOs. 

57% 

70% 

+13%* 

42% 

50% 

+8%* 

Information about the use of SLO components. 

64% 

74% 

+1 0%* 

44% 

50% 

+6%* 

Resources supporting the use of student baseline data. 

56% 

72% 

+16%* 

35% 

43% 

+8%* 

Guidance in selecting learning content for SLOs. 

50% 

76% 

+26%* 

32% 

50% 

+1 8%* 

Guidance in selecting research-based instructional strategies 
for SLOs. 

45% 

63% 

+1 8%* 

27% 

37% 

+1 0%* 

Recommendations for selecting pre/post assessments for 
SLOs. 

44% 

71% 

+27%* 

29% 

48% 

+19%* 

Valid and reliable approaches to setting growth targets for 
SLOs. 

42% 

60% 

+18%* 

28% 

40% 

+12%* 

Capacity Building 

1 need support in... 

Gaining timely access to student data. 

51% 

55% 

+4% 

55% 

47% 

-8%* 

Analyzing student data for action. 

44% 

55% 

+1 1 %* 

53% 

46% 

-7%* 

Using pre/post assessments. 

48% 

49% 

+1% 

48% 

38% 

-10%* 

Receiving SLO feedback from school or district 
administration. 

67% 

55% 

-12%* 

66% 

50% 

-16%* 


Note: * indicates statistically significant differences at the 95% confidence level. Percentages represent a composite 
of strongly agree/agree. 


TPE Implementation with Different Teacher Populations 

Different populations of teachers vary in their response to the TPE implementation. This finding is 
iiiustrated by comparing groups of teachers based on teaching experience, and groups of teachers 
based on their schooi ievei. 
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In the survey, 63% of teacher respondents have ten or more years of experience. These veteran 
teachers are critical to the success of any school wide reform, including the implementation of the TPE. 

As the survey data show, veteran teachers are more likely to agree than disagree about the positive 
potential of TPE (see Table 3, column 1 0+ Years). Veteran teachers are likely to agree that the 
expectations for TPE are clear (48% agree versus 32% disagree), that the TPE frameworks and 
processes recognize the scope of an educator’s roles and responsibilities (43% agree versus 35% 
disagree), that TPE frameworks respect educators’ professional knowledge and skills (51 % agree versus 
32% disagree), that the TPE frameworks use valid measures instruments (41 % agree versus 34% 
disagree), are led by qualified observers (58% agree versus 21 % disagree), provide useful feedback to 
teachers (46% agree versus 28% disagree), lead to improved decisions related to instructional 
approaches (45% agree versus 28% disagree), and inform professional development plans (43% agree 
versus 25% disagree). 

On the other end of the experience spectrum, teachers who are relatively new to teaching with 0 to 3 
years of experience are even more positive than veteran teachers about TPE and its potential impacts. 
On each item listed in Table 3, newer teachers have higher expectations and are more likely to agree 
with statements about TPE than are the veteran teachers. 

These data show that principals already have a base of teachers who are more positive than negative 
about the teacher evaluation system, and newer teachers coming into the system are even more likely to 
view TPE positively. This positive group gives principals the social capital within schools to keep driving 
the quality of the work related to SLOs, observations, and the other instructional reforms occurring in 
schools. 


Table 3: Responses of New Teachers Compared to Those with Ten or More Years of 
Experience 



0-3 Years 

10+ Years 

Agree 

Undecided 

Disagree 

Agree 

Undecided 

Disagree 

The expectations are clear for teachers and 
principals in the TPE system.* 

60% 

18% 

22% 

48% 

20% 

32% 

TPE recognize the scope of an educator’s roles 
and responsibilities.* 

56% 

21% 

23% 

43% 

21% 

35% 

TPE frameworks and processes respect educators’ 
professional knowledge and skills.* 

67% 

14% 

18% 

51% 

17% 

32% 

TPE frameworks and processes utilize validated 
observation measures/instruments.* 

58% 

19% 

23% 

41% 

24% 

34% 

TPE frameworks and processes are conducted by 
observers/evaluators qualified to do the 
evaluation.* 

75% 

12% 

13% 

58% 

21% 

21% 

TPE frameworks and processes provide useful 
feedback to teachers.* 

64% 

18% 

18% 

46% 

26% 

28% 

TPE frameworks and processes lead to improved 
decisions related to instructional approaches.* 

63% 

20% 

17% 

45% 

28% 

28% 

1 believe the results of my evaluation will be used 
to inform my professional development plan for 
next year.* 

56% 

24% 

19% 

43% 

32% 

25% 


Note: * indicates statistically significant differences at the 95% confidence level. Agree \s a composite of strongly 
agree/agree. Disagree \s a composite of strongly disagree/disagree. 
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Based on survey responses, elementary school teachers are strongly positive about the TPE 
implementation in their schools. Elementary teachers are more positive than middle school teachers 
who in turn are more positive than high school teachers. In survey responses, elementary school 
teachers are more likely to agree that expectations are clear (54%) than teachers in middle schools 
(48%) or high schools (45%); and that TPE is based on continuous improvement (66% compared to 
59% for middle school teachers and 52% for high school teachers). Similarly they are more likely to 
agree that TPE recognizes the scope of an educator's roles and responsibilities (50%) than teachers in 
middle schools (45%) or high schools (39%), and that their school’s support of the implementation of the 
TPE system is helpful (60% compared to 52% for middle school teachers and 50% for high school 
teachers). 

Elementary teachers are more likely to indicate that the TPE system respects educators’ professional 
knowledge and skills (ES 59%, MS 51 %, and HS 45%), provide a summative rating of educator 
performance (56%, MS 48%, and HS 43%), provide useful feedback to teachers (ES 55%, MS 47%, 
and HS 41 %), encourage reflection on instructional practices (ES 71 %, MS 64%, and ES 59%), and lead 
to Improved decisions related to Instructional approaches (ES 55%, MS 45%, and HS 39%). 


Table 4: Teacher Perceptions of TPE by School Level 



Elementary 

Middle 

High 

Systemic Context 

Agree 

Undecided 

Disagree 

Agree 

Undecided 

Disagree 

Agree 

Undecided 

Disagree 

Expectations are clear for 
teachers and principals in the 
TPE system.* 

54% 

20% 

26% 

48% 

21% 

32% 

45% 

19% 

37% 

TPE is based on principles of 
continuous improvement.* 

66% 

22% 

12% 

59% 

23% 

18% 

52% 

25% 

24% 

TPE recognizes the scope of an 
educator’s roles and 
responsibilities.* 

50% 

21% 

29% 

45% 

21% 

34% 

39% 

22% 

39% 

My school’s support of the 
implementation of the TPE 
system is helpful.* 

60% 

22% 

18% 

52% 

25% 

23% 

50% 

24% 

25% 

Quality of TPE Frameworks and Processes 

Teacher evaluation frameworks and processes... 

Respect educators’ professional 
knowledge and skills.* 

59% 

16% 

25% 

51% 

17% 

32% 

45% 

17% 

38% 

Provide a summative rating of 
educator performance.* 

56% 

22% 

22% 

48% 

23% 

28% 

43% 

23% 

34% 

Provide useful feedback to 
teachers.* 

55% 

24% 

21% 

47% 

24% 

29% 

41% 

25% 

35% 

Encourage reflection on 
instructional practices.* 

71% 

16% 

13% 

64% 

18% 

18% 

59% 

18% 

23% 

Lead to improved decisions 
related to instructional 
approaches.* 

55% 

25% 

20% 

45% 

28% 

27% 

39% 

27% 

34% 


Note: * indicates statistically significant differences at the 95% confidence level. Agree \s a composite of strongly 
agree/agree. Disagree \s a composite of strongly disagree/disagree. 
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School, District and State Support 

Implementing a multi-faceted reform like TPE requires a lot of training and support. In surveys and 
interviews, educators say that the key support they receive is from their own district or school. 

Most principals (77%) report getting helpful support from the district and from their own school (88%). 
Some 44% find state support helpful and 45% find school community support helpful. Teachers give 
lower ranking to state support (25%); somewhat higher ranking to district support (40%); and still higher 
ranking to school support (55%). Only 27% of teachers find school community support helpful. Teachers 
report that they are less likely to have direct interaction with the state regarding TPE, while district level 
interaction is varied— some districts provide extensive TPE related support, others leave it up to the 
schools to train and support teachers. Note that 88% of principals and 55% of teachers indicate that 
their schools’ support of TPE implementation is helpful. Bear in mind that the principals generally have 
the lead responsibility for providing this support. 


Figure 3: Teacher and Principal Views on the Support for TPE 
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of the TPE system is 
helpful.* 


The school 
community’s support 
of the Implementation 
of the TPE system is 
helpful.* 


Note: * indicates statistically significant differences at the 95% confidence level. Agree \s a composite of strongly 
agree/agree. Disagree \s a composite of strongly disagree/disagree. 


MSDE plays a pivotal role in guiding TPE policies in Maryland and has ongoing meetings with 
superintendents about TPE. Given the state’s local control structure, however, districts decide on the 
exact form of TPE to adopt and on the implementation approach. Asked how effective MSDE has 
been in helping them with TPE implementation, superintendents are generally positive. Some 
describe what they perceive as earlier missteps by MSDE, which they largely attribute to earlier turnover 
in top state leadership positions. But most describe MSDE support this year as beneficial. 
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"MSDE was an excellent resource. They did a good job in giving examples and explaining what 
an SLO is expected to do." 

-Superintendent 

"A strength is the state's willingness to make adjustments." 

-Superintendent 

"MSDE is on it. They've been very, very receptive." 

-Superintendent 


Instructional Dialogue 

The TPE process is designed to prompt educators to continuaiiy improve by using data to refiect on 
their own instructionai practices and by engaging in meaningful instructional dialogue with colleagues 
and supervisors. In interviews, superintendents express a clear priority on strengthening the 
dialogue between principals and teachers, saying that that dialogue resulting from the TPE process 
can be used to help teachers improve their practice. They acknowledge that this intensive focus on 
instructional dialogue is new for some principals, and it will improve, with practice, over time. 


"We are going to have deep, moving conversations about children." 

-Superintendent 

"We will put a heavier emphasis on collaboration across content areas, with multiple teachers 
working on the same SLOs." 

-Superintendent 

"TPE will have a positive overall impact. It'll affect dialogue and there will be some clumsy 
moments, but it'll make a difference in what the conversation is about. We are going to use the 
data to engage the interest of principals and teachers. Teachers and principals are talking about 
groups of kids, telling us in detail about the kids who are failing, and examining what they are 
trying to do to help the students." 

-Superintendent 


Principals, especially, report seeing a shift toward more teacher reflection on practice and more 
instructionally focused conversation in their schools. Figure 4 shows that 72% to 78% of principals 
agree that reflection on instructional practices has deepened and there was more instructionally focused 
dialogue with colleagues and supervisors. 


"I think the dialogue with my teachers has changed because I don't have to force my teachers to 
look at the data. They forced themselves to look at the data because they know they have to use 
that data to drive their instructional practices." 

-Principal 
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"TPE changes teachers' practice because of the paradigm shift. All are more reflective." 

-Principal 

“It's beneficial. Teachers talk more about quality questioning, differentiated instruction, and 
student engagement. The impact can be huge." 

-Principal 


Both teachers and principals acknowledge a learning curve associated with developing an 
understanding of the various components of TPE. They indicate it takes time to begin using the TPE 
tools to improve instructional dialogue, instructional practices and instructional outcomes. Teachers are 
more mixed regarding a shift in instructional dialogue with 39% to 44% of teachers agreeing that 
reflection deepened or that there was more instructional dialogue. A comparable percentage of teachers 
disagrees that there was more reflection or instructional dialogue. 


Figure 4: Teacher and Principal Views on This Year’s Experience 
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Note: * indicates statistically significant differences at the 95% confidence level. Agree \s a composite of strongly 
agree/agree. Disagree \s a composite of strongly disagree/disagree. 


Teacher Ratings 

The topic of teacher ratings in the new system elicits a mix of responses from interviewees. Many feel 
that ratings are not going to change very much from the previous evaluation system. For example, a 
union leader notes, “Any change? Probably not. Most principals hesitate to give 3s and most hesitate to 
give 1s or Os. They're staying in the safe zone." Other interviewees feel that achieving a highly effective 
rating wili be more difficult than in the past. 


“With new ratings, teachers feel consternation over being merely effective. You have to talk them 
off the ledge. Getting effective is good." 

-Superintendent 
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“My fear is we have a number of people who see themselves as highly effective and who have been 
told they are highly effective. When they will be merely effective, they will be crushed. ” 

-Union Leader 

“I expect the preponderance of teachers will be effective. There is a rub with that: Teachers are 
used to the highest rating. Principals can identify the true stars and it is a few of their teachers. 
They can identify who is scraping by, and they put most teachers in a middle category." 

-Superintendent 


Looking at districts where both administrators and teachers participated in interviews, districts faii into 
three groups. Some Maryland districts view the TPE system as part of a broader approach to improving 
instruction. Other districts indicate that dealing with the demands of a new evaluation system is currently 
superseding the more sophisticated instructionai uses of TPE. A third set of districts seem to view TPE 
as a compliance activity to meet state and federal mandates rather than as a means to improving 
instruction. 

Districts had not yet produced finai ratings at the time of this data coiiection, so teachers were waiting to 
see what their final ratings would be. Teacher expectations for TPE ratings differ. This difference is 
largeiy a function of the degree to which their district is connecting the TPE process to the district’s 
instructionai system. In districts that are making this key connection, teachers have fewer concerns. By 
contrast, in districts that appear more focused on complying with mandates, a number of educators say 
they do not know what to expect or they have concerns about how results would be used. They are 
concerned that they might receive poor ratings, or afraid of losing their jobs. Teachers in these districts 
would like to see more responsiveness to the morale of teachers, and more participation in TPE 
implementation decisions. 


"Teachers really fear that they may lose their jobs. There is a lot of paralyzing fear among 
teachers especially since they do not know what the test will look like. We can get them focused 
on the positives." 

-Superintendent 


"Teachers are overwhelmed, insecure, frustrated." 


-Teacher 


On the other hand, in districts that involved teachers in the TPE process and that view the TPE system 
as part of a broader approach to improving instruction, a number of educators feel that the potential and 
perhaps the promise of the new evaluation system will yield an evaluation that could help teachers help 
students. Teachers in these districts indicate that the TPE system is fairer than prior evaluation systems, 
and that being held to a higher standard of teaching would help them improve their craft. A number of 
administrators suggest that the new TPE system wouid iead to broad improvements in instruction. 


"TPE is more teacher oriented. . .more teacher developed. . .more equitable." 

-Teacher 
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"It [TPE] sets high expectations for teachers just as for students." 

-Teacher 

"If TPE plays the way it should, it will catapult everything - we'll be providing instruction and 
experiences so the children can grow more. In my career, I have never seen anything like this." 

-Superintendent 


Summary 

On many issues, the perceptions of Maryland’s frontline educators are positive about the potential of 
TPE. As they have more experience in implementing the new evaluation system, teachers and principals 
indicate that their confidence and skill levels are growing. They also raise concerns that need to be 
addressed if the quality of implementation is to increase. 
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Chapter Four: Common Issues in Implementing TPE 


This discussion buiids on findings from chapter three and highlights additional issues that affect the 
implementation of TPE. The issues are the pace of the reform; school and district capacity to integrate 
initiatives; developing consist practice and common language; principals’ capacity and support; 
teachers’ capacity and support; TPE data and technology; and observations and the observation 
process. 

Pace of Reform 

By definition, changing to a performance-based evaluation system gets people’s attention. When the 
changes are taking place on a statewide scale, issues of time and capacity are on center stage. They 
affect how educators view the pace, consistency and quality of implementation. Moreover, how these 
issues are addressed affects how effective and valued the new system will be. The context of TPE 
implementation in Maryland is driven by timetables laid out in the state’s Education Reform Act of 2010, 
its Race to the Top application and its ESEA Flexibility waiver. 

Interviewees across role groups suggest that the pace of 
implementation has been unrealistic. Superintendents, 
principals and teachers speak of the time required to put new 
TPE processes in place. Each process involves a learning 
curve, and school staffs say the time pressures are 
compounded by other reforms happening simultaneously. As 
noted in chapter three, schools and districts that opted for a pilot have an advantage, but even there 
staff speak of too much change too quickly. Some believe that the stresses of rapid implementation 
cloud people’s ability to see the potential payoffs of the reform. 


"Teachers are always in the 
third year of a five-year plan." 

-Teacher 


"The timetable was too fast. That is a public policy decision. We all know the misalignment 
between public policy and the real world." 

-Superintendent 

"The timeline has been so unrealistic in terms of appreciating the magnitude of changes we have 
to make. So much learning has to go on here; we have not had the time." 

-Superintendent 


One effect of the pace of reforms is educators feel they do not have time to build in all the supports they 
need at the school and district levels to help teachers with the reforms. Interviewees emphasize that 
effective implementation depends on developing know-how and support systems at district, 
school and classroom levels. A veteran teacher, noting how reforms often fail to prepare school level 
educators for new requirements and roles, comments, “Teachers are always in the third year of a five- 
year plan.” 


"A limitation is no one has done evaluation this way before. So we are figuring out how to do it." 

-Superintendent 
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"The infrastructure to support implementation needs to be strengthened." 

-Teacher 

"With so many initiatives at the same time, we need to slow down and get things right before 
using them for accountability." 

-Teacher 


School and District Capacity to Integrate Initiatives 

A challenge for districts and schools is that they need to integrate a number of initiatives into a 
strategically coherent instructional improvement process. Transitioning to TPE is a massive undertaking, 
but TPE implementation is occurring concurrently with the change to Maryland’s College and Career- 
Ready Standards, and a change in the statewide assessment system. Superintendents see the 
challenges of marshaling district resources so that frontline educators can effectively make the 
connections between SLOs, observations and the new Maryland College and Career-Ready 
Standards. 

The job of integrating the initiatives so they make sense as pieces of a broader instructional strategy 
often falls to superintendents, curriculum leaders and principals. Where they see each piece of the 
reform as part of a whole instructional improvement strategy, they are better able to prioritize time and 
tasks, and help teachers make sense of the new information and processes. Some districts have been 
working on these changes for several years and are able to make coherent connections between TPE, 
new standards, new assessments and improved instruction. In other districts, interviewees indicate that 
they are struggling with making these connections. Universally, superintendents agree that this critical 
work is proving to be a significant undertaking. 


"You have to talk to teachers and principals about all the evaluations, and how they fit with the 
Common Core. If you do not connect the pieces, you end up with silos. They are so consumed 
with day to day work. You have to paint the picture for what this whole thing looks like." 

-Superintendent 

"We have four deliverables: The Common Core, teacher and principal evaluation, PARCC, and 
longitudinal data tracking. They all require bringing people up to speed and building a 
curriculum that is aligned with the standards. They also require trust; this is a people 
business." 

-Superintendent 

"We started looking at all the pieces separately - we had not heard of SLOs - but now we are 
talking about all the pieces: connecting Common Core to observations to improved SLOs and 
improved instruction. It is incumbent on us to make those connections. We did not at first, but 
we're trying to make those connections now." 

-Superintendent 
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Developing Consistent Practice and Common Language 

A concern that emerges in interviews is that TPE implementation is inconsistent, a problem that 
stems in part from the lack of a common understanding of the framework and processes. TPE 

approaches differ from district to district, school to school, sometimes even within schools. Teachers 
report inconsistent focus on differing components of the reform as well as inconsistent support for 
learning and implementing new processes and practices. Some of this inconsistency can be attributed 
to the many educators who have not had prior experience with TPE components and are both 
implementing and learning as they go. Where principals experience gaps in capacity or support, 
implementation of TPE also tends to be inconsistent. 


"There's real inconsistency in principals' interpretation of the rules." 

-Teacher 

"There are lots of inconsistencies in implementation. Even by subject area, there are 
inconsistencies by level." 

-Teacher 


"It was disjointed. There was a focus on SLOs and Danielson for a week or two, but the principal 
could not answer questions. Now, in the last two weeks, the principal says we need to get this 
done. " 

-Teacher 


In some districts, educators are still developing a common language for talking about components of the 
TPE system and how they can be used for instructional improvement. Lack of common language 
contributes to inconsistent understanding from classroom to classroom on what constitutes 
effective instruction and how it will be assessed. Some districts report working diligently with 
teachers and principals to develop shared understandings. In other districts, it was clear from the 
interviews that administrators and teachers have not yet developed these shared understandings. 


"We need to work towards a common understanding of indicators and terms." 

-Superintendent 

"Have a common language when supervisors and principals talk. Be cohesive. I almost want to 
say: re-do the procedures." 

-Teacher 

"We worked very hard with the association to develop shared understandings of fair evaluation, 
licensed observers, and training needs." 

-Superintendent 


Part of the inconsistency is also driven by the need for districts and schools to build better 
communication with teachers. Two-way communication provides the foundation for constituency 
building and collaboration, but interviewees report that communication is functioning differently at state, 
district and school levels. 
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Superintendents are most likely to be in dialogue with state TPE officials, and they are positive in 
describing the state’s TPE-related communication. Similar to their comments about state-to-district 
support as described in chapter three, superintendents indicate that state-level communication is strong 
and improving overtime. 


"They are very receptive when we give feedback. They have a willingness to look at and address 
problems we identify and our concerns." 

-Superintendent 

"There has been an attempt to provide communication to all the stakeholders. They regularly 
send communications which tend to be lengthy, but there is an effort to keep the lines of 
communication open. 1 see that as a strength." 

-Superintendent 


The challenges of communication, though, are pronounced in some districts and schools. In some 
districts, teachers report that they do not have collaborative communication with administrative leaders 
in their district. Gaps in communication create impediments to the implementation of TPE and 
make it more difficult for teachers to contribute to efforts to strengthen TPE implementation. The 
resultant effect is that teachers then feel disempowered or feel left out of a process they do not 
understand as well as they would like to. 


"There is no formal avenue for teachers to get together to help make changes." 

-Teacher 


"The message is inconsistent from school to school based on how that administrator implements 
it." 


-Union Leader 


"Principals and supervisors need to ask teachers for their opinions about implementation. That's 
when teachers feel valued." 

-Union Leader 


In some districts, teachers report close working relationships with administrators, and have ongoing 
meetings to maintain lines of communication and solve problems. When the communication is strong, 
teachers report more trust in the TPE process and they report that collaboration increases. The quality 
and level of communication between district and union leaders matters. For the second 
consecutive year, there is a consistent pattern in interview responses: where there is better 
communication between and among district and union leaders, there is better implementation of TPE. 


"We have real dialogue and TPE is why it's happening." 


-Union Leader 
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"We have a fortunately good working relationship with our union. Our union knows that we get 
that evaluation is to improve instruction and improvement, and it is not something bad. They 
get that and it helps." 

-Superintendent 

"The superintendent is constantly asking for feedback. 1 constantly meet with my staff, and when 
1 meet with teachers we discuss the process." 

-Principal 


Principals’ Capacity and Support 

One of the goals of TPE is to support the role of the principals as the instructional leaders in the school 
so that they, in turn, can focus attention on those teachers who need the most support. Some 
principals indicate that the increased emphasis on being in classrooms working with teachers on 
SLOs and observing their instruction is leading principals to be more engaged in the school as an 
instructional leader. Some principals acknowledge that prior to TPE they had not been in the 
classrooms as often as they are now; for other principals, the frequency of classroom visits did not 
change but the focus of those visits has changed. 


"Because of TPE, I've become a more present instructional leader, not a manager. I visit every 
classroom daily. Teachers, paraprofessionals and students are now used to the principal and 
assistant principals being in the classrooms." 

-Principal 

"TPE forces us to look at how we engage kids, and helps us improve the quality of our questions. 
When 1 go to classes, 1 am more focused." 

-Principal 

"It's a struggle, but I think principals were ready. They feel an increasing responsibility to do 
more touching base with teachers along the way. It's a crunch on time, but they think the 
conversations have strengthened the relationships with teachers. This system has helped 
principals identify those teachers that need more attention" 

-Superintendent 


Interview findings suggest that many principals perceive the need to improve their instructional 
leadership skills. The new evaluation system and all its components succeed or fail with the capacity of 
the principal to guide the process. Superintendents cite capacity building for principals as key in the 
success of implementation. They note that much of the work of SLO reviews and teacher observations 
falls on the principal and the principal’s team. Principals report that while they are learning the new 
systems, everything takes longer. Principals who implemented previously in the pilot test report they are 
more efficient in managing the process compared to when they started. 


"We always talk about student achievement gaps: this is about principals' gaps and their comfort 
level." 


-Superintendent 
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"Who is really ready for any of this? Principals will only be ready if we can continue our 
professional development initiatives." 

-Superintendent 

"Implementation took a lot of time. 1 had to make sense ofTPE and then focus the teachers with 
guidance. " 

-Principal 


Some teachers observe that the demands on principals are too great. Teachers worry about how 
much help they can expect from principals who are overwhelmed by the time demands of new 
observation and SLO processes, given already busy workloads. When principals are overwhelmed, 
teachers do not receive the support they need, and many processes in schools that rely on principal 
input can suffer from limited attention. 


"For principals, the hardest problem is the workload. Principals that piloted last year are more 
comfortable, but it's still very time consuming." 

-Union Leader 

"I think the administrators are feeling the push and the crunch more than us. They are more 
stressed than we are." 

-Teacher 


"Reality wise, it's not doable for the administrators." 


-Teacher 


Some principals treat the different components of the TPE system as separate tasks, and the result is 
that the disconnected tasks can then be overwhelming. Principals need to be clear about what their role 
is with SLOs and observations, and how the school leadership structure supports those roles while also 
supporting teachers. Where roles are not clearly defined, teachers complain about the lack of support. 
Teachers report that the overwhelmed principal can cause problems for the staff, for instance by giving 
teachers confusing messages or by becoming hard to deal with. 


"We were told by the principals, 'Don't get too stressed, no one knows what is going on.' Then, 
at the end of the school year, we were told it all counted." 

-Teacher 

"With TPE and all the other initiatives, principals are stressed out and burned out. It's too much 
and they're telling teachers, 'it's going to be done my way."' 

-Teacher 
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Many interviewees are concerned about the readiness of principals to conduct fair and reliable 
observations. They have a number of related concerns regarding principal capacity. Some teachers are 
concerned about inter-rater reliability, noting that different teachers in their school seem to receive 
different ratings for the same work. Other teachers say that they are not receiving the quality of training 
they need and wonder if the principals receive sufficient training. For other teacher interviewees, the 
concern is with their principal's knowledge of the observation rubric and teaching pedagogy. Finally, 
some noted that it is unclear what principals look for in an observation. 


"The whole issue of inter-rater reliability is a big hurdle. It'll be a problem if it gets challenged." 

-Superintendent 

"One complaint 1 hear from teachers is that some observers don't tell what they are looking for. 
You cannot see all the domains with drop-ins. So teachers are getting penalized. Principals need 
more training on it." 

-Teacher 

"Our supervisors are responsible for way too much. Some supervisors are not experts in the 
subject areas they observe. Sometimes it's so overwhelming. Sometimes it is difficult for teachers 
when we feel we know more." 

-Teacher 


Principals cite specific areas in which they feel they need more support. Survey findings show that 
the kinds of support principals need Include: using pre/post assessments (49% agree); and more SLO 
feedback (55% agree). Principals also would like additional support for using Maryland's College and 
Career-Ready Standards (63% agree), using data to strengthen school Improvement planning (64% 
agree), and using data to Improve Instructional supervision (70% agree). 

Figure 5: Principal Views on Capacity Building 
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Many of the needs principals cite relate to SLOs. For principals to succeed with the work SLOs require, 
they need to marshal expertise in their building to establish a cohesive SLO strategy. Together, 
principals and teachers need to develop working knowledge in many areas: knowing what should be 
assessed, developing meaningful learning objectives tied to Maryland's College and Career-Ready 
Standards, knowing how to do pre- and post-testing well, being able to accurately Interpret assessment 
results for student Instructional needs, and knowing how to rate and interpret SLO quality to understand 
gaps in teaching that may need to be strengthened by providing additional Instructional support or 
professional development. These are important areas where principals need a high level of competence 
to achieve the promise of the TPE system as a system that drives continuous instructional Improvement. 

Teachers’ Capacity and Support 

In districts that have significant involvement of teachers in TPE implementation, teachers report that the 
ongoing dialogue with district administrators means that problems are Identified and dealt with on a 
continuous basis. While not minimizing the challenges, they talk of processes to deal with the challenges 
to get teachers the support they need. In other districts where teachers report having less opportunity 
for such involvement, teachers voice the need for more of everything— dialogue to understand the TPE 
system, training on its components, professional development to Improve Instruction — but they mostly 
want a better system to address problems and their need for support. 

Across districts, teachers say they need more training on the components of TPE and also need 
professional development to plan and deliver instruction more effectively. In Interviews, most 
teachers do not find their level of training on components of TPE to be sufficient. Moreover, they feel and 
understand that there’s a difference between receiving training on the components of a new evaluation 
system and getting the professional development needed to plan and deliver instruction more 
effectively. 


"Teachers and principals are primarily receiving training on TPE; they are not receiving 
leadership development and professional development.” 

-Union Leader 

"Too little has been invested to make it [TPE] work. There needs to be a substantial investment 
in training. ” 

-Union Leader 

"Training on how to plan lessons differently or how to use the MSDE site? That's not 
happening. ” 

-Teacher 


Teachers indicate that additional training and professional development is needed to use the 
observational frameworks effectively. Teachers in the focus groups indicate that they may know what 
is listed on the observational frameworks, but not necessarily understand how to incorporate individual 
teaching components into everyday instruction. As a result, they feel that the emphasis on which 
components are critical for dally Instruction can be diminished, with the result that the observational 
rubrics can easily become checklists. 
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"Teachers need concrete and standardized professional development before next semester to help 
us understand the Danielson framework." 

-Teacher 

"INe need a standardized professional development for all teachers on Danielson at beginning of 
year. A lot of really good teachers feel inadequate in their job." 

-Teacher 

"We were given examples of the domains and the indicators. You can focus on the areas you need 
to improve. " 

-Teacher 


It is not just training on the components of TPE that teachers want. In surveys, teachers identify a 
number of areas where they need more support. Teachers indicate that they need more help using 
Maryiand's Coiiege and Career-Ready Standards (55% agree), more SLO feedback (50% agree), more 
help using data to strengthen school improvement planning (48% agree), and using data to improve 
instructionai practice (48% agree). 


Figure 6: Teacher Views on Capacity Building 
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These areas of support that teachers identify mirror the areas of support principals need. Teachers are 
struggling with incorporating Maryland’s College and Career-Ready Standards into their lesson plans 
and tying their instruction and assessments to those standards. They also need support using data from 
TPE to improve their instruction, suggesting they need more support creating high quality SLOs, 
interpreting data from those SLOS, linking that data to their instructional practices, and understanding 
how to improve instructional practices in light of that data. Where schools can help teachers with these 
activities, they will begin to develop a TPE system that supports continuous instructional improvement. 
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The need for additional support to understand and implement TPE is more acute in Maryland's high 
need districts, where student eligibility for free or reduced price lunch is between 50 and 85%. 
Educators in these districts express consistently stronger needs for support than do their peers in 
wealthier districts. 


Table 5: Teachers’ Responses by District Levels of Poverty 



Low Poverty 

High Poverty 

Systemic Context 

Agree 

Undecided 

Disagree 

Agree 

Undecided 

Disagree 

Gaining timely access to student data.* 

45% 

16% 

39% 

52% 

13% 

35% 

Analyzing student data for action.* 

44% 

14% 

43% 

50% 

13% 

37% 

Using pre/post assessments.* 

36% 

14% 

51% 

43% 

13% 

44% 

Using Maryland's College and Career-Ready 
Standards.* 

54% 

19% 

27% 

58% 

18% 

24% 

Receiving SLO feedback from school or district 
administration.* 

46% 

20% 

35% 

59% 

17% 

24% 

Using data from SLOs and observations to strengthen 
school improvement planning.* 

45% 

22% 

33% 

56% 

20% 

24% 

Using data from SLOs and observations to make 
improvements in my instructional practices (for 
teachers only).* 

44% 

18% 

38% 

55% 

17% 

28% 


Note: * indicates statistically significant differences at the 95% confidence level. Agree \s a composite of strongly 
agree/agree. Disagree \s a composite of strongly disagree/disagree. 

Low Poverty = districts with 18-30% of students receiving free or reduced-price lunch (FRL) in 2013. 

High Poverty = districts with 52-84% of students receiving FRL in 2013. 

TPE Data and Technology 

Many educators describe the increased amount of data that the TPE process generates. There are data 
generated about the TPE process itself— for SLOs: documenting the SLO components and targets, the 
mid-year review, and end of year review; and for observations: documenting the pre-observation 
meetings, the observations, and the post observation meeting; and for the overall ratings: documenting 
the review of the prior year professional development, the review of this year’s ratings and needed 
improvements, and the professional development plans for summer and next year. All of these data 
need to be maintained for each teacher to ensure a fair and consistent TPE process. In short, there is a 
need for a data system that makes these data easy to add, edit and retrieve. 

Each component of the TPE process, in turn, generates data about students (pre- and post-test data in 
SLOs, differentiated groupings, instructional strategies used) and teachers (actual observation data). 
These data need to be maintained so teachers and supervisors can easily access and analyze student 
progress, link that progress to learning content and instructional strategies, diagnose student learning 
deficiencies and issues, and understand how to adjust instructional content and strategies to help 
students improve. 
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Districts and schoois need both a robust data system that can handle the volume and type of data 
generated by the TPE process, link these data to other assessment and administrative data, and 
facilitate the analysis of the data so teachers can interpret what students need and make changes to 
their instructional focus and delivery to meet those needs. Educators in many districts indicate that 
they are struggling with the technology needed to manage TPE-related data and to facilitate 
teacher access to data; some are maintaining their information with paper and pencii. 


'We do not have the capacity in this county to provide sound instruction for students and 
manage the TPE system. Our technology is abysmal, but it's not the worst in the state." 

-Superintendent 


"You were not given any templates. It's a paper process." 

-Teacher 

"Some schools kept all their data on Word. Different departments in my school used different 
formats to collect data." 

-Teacher 


When it comes to accessing and using data, both principals (55%) and teachers (47%) agree that they 
need more support in gaining timely access to student data, suggesting that their data and assessment 
systems have limitations in giving them the data they want in a timely fashion. Both principals (55%) and 
teachers (46%) also agree that they need more support in analyzing student data for action, suggesting 
that they are not always sure how to analyze or interpret data, or decide what to do based on the data. 
These are important issues because a new evaluation system wiii fali short of its intended goals if it 
proves to be data rich, but information poor. 


Table 6: Teacher and Principal Views on Capacity Building 


Capacity Building 

1 need support in... 

Agree 

Undecided 

Disagree 

Gaining timely access to student data.* 

Principals 

55% 

9% 

35% 

Teachers 

47% 

15% 

38% 

Analyzing student data for action.* 

Principals 

55% 

7% 

38% 

Teachers 

46% 

14% 

40% 


Note: * indicates statistically significant differences at the 95% confidence level. Agree \s a 
composite of strongly agree/agree. Disagree is a composite of strongly disagree/disagree. 


In interviews, teachers express some skepticism about what impact data generated out of TPE will have 
on student outcomes, expecting it to take some time before they see results. Superintendents and 
principais are generally positive about the immediate potentiai for using data to improve instruction. For 
this to be true, both teachers and principals need additional understanding about what TPE- 
generated data mean and how to analyze and use this information to improve instructional 
practice. 
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"TPE creates so many data. I've been trying to figure out what this data really tells me about 
how people are doing and whether they are really making progress with student learning. It's 
hard to figure out, but it's important. 1 wish we had done that a long time ago." 

-Superintendent 

"I would like to have as much extra time as possible, and 1 plan to create more faculty time and 
build data analysis into teachers' daily work." 

-Principal 


Implementing Observations 

Teacher observations, similar to SLOs, are an evidence component of the TPE process. The balance of 
this chapter outlines findings about the implementation of observations. 

Both teachers and supervisors feel comfortable with the idea of using observations in the TPE 
process. For many, this is because the observation process in the TPE system adopted by their districts 
is familiar. It is relatively similar to what was already in place in the districts. They value the observations 
as a tool for reflection, and as a means to have a more engaged dialogue around effective teaching. 


"The pre- and post-observation conferences have us much more focused on what is happening in 
the classrooms. 1 think those conversations will get better as teachers understand it better." 

-Principal 


"The observations make you a more reflective practitioner." 

-Teacher 

"The new system has detailed rubrics for observations and professional practices. They eliminate 
subjectivity." 

-Teacher 


When asked about the observation process, principals are clear in their survey responses that they 
see themselves as qualified to conduct observations. Ninety percent of principals agree that 
observations are conducted by observers who are qualified to do the evaluation. About three-quarters of 
principal survey respondents agree that TPE utilizes validated observations measures/instruments. 

Teachers have lower rates of agreement about the qualifications of observers and the use of validated 
instruments for observations. Forty-four percent of teachers agree that TPE utilizes validated observation 
measures/instruments. When asked if they agree that TPE observations are conducted by observers 
who are qualified to do the evaluation, 61 % of teachers agree. 
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Figure 7: Teacher and Principal Views on Observations 
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For these two questions, there is approximately a 30% difference between principal and teacher 
response rates. While there are a considerable number of teachers and principals who agree on these 
two statements, teachers are more likely to disagree with these statements than are principals. High 
rates of disagreement within a school on the quality of the observation process and instruments may 
erode trust in the process and iimit the desired impact on instructionai quaiity. 

Although many interviewees have more experience with an observation process than with SLOs, 
they are still building their understanding of the instruments and the high stakes nature of the 
observations. Most feel that the way to learn the process is to do it. Teachers worry about the use of an 
observationai rubric in their formai evaiuation, fearing that being observed on one bad day may affect 
their score for the year. 


"I feel like I am learning as we go. We had online modules in Danielson, but I still feel I do not 
have the understanding 1 should have, and the teachers really do not understand it. They have 
seen it but they have not fully learned it. They don't know the language, the indicators, or what 
to do to hit the mark." 

-Principal 

"It has taken us one and a half years to feel we're doing the formal observation process right." 

-Superintendent 

"That's one day. Ifl am not highly effective in every single area in that day, 1 am not highly 
effective in that year... 1 don't think one day is an appropriate representation of what 1 look like. 
It's not a fair representation of my abilities. It is a set up so no one is highly effective." 

-Teacher 
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Teachers indicate that the implementation of the observation process varies between and within 
schools. Some teachers note variations in implementation practice by department— different 
departments within a school may have different expectations for what they emphasize in an observation. 
Some teachers note variations in implementation practice by conferences— some schools do both a pre- 
conference and a post-conference, but others do not give both conferences or different people conduct 
the conferences with different approaches and criteria. In some districts, teachers report their principal 
does a good job integrating the observation process into a holistic approach to instructional 
improvement, while other teachers In the district describe principals who approach the observations in a 
mechanistic way. 


"I had different people give me the pre-observation meeting and the post-observation meeting." 

-Teacher 

"When it happens, the pre- and post-conferences made me feel like a professional." 

-Teacher 

"It's frustrating that in some departments the bars are so high that it's not realistic, but in some 
departments the bars are low." 

-Teacher 


The credibility and usefulness of the observation process require fidelity of implementation and serious 
examination of practices related to teachers’ understanding of pedagogy and the quality of their delivery 
of instruction. While some districts are working to maximize the potential of teacher observations, others 
need further assistance to move more effectively in this direction. 
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Chapter Five: SLO Implementation in Maryland 


Effective statewide implementation of SLOs is essential to Maryland’s instructional Improvement and 
accountability strategy. As a foundation of the new Teacher and Principal Evaluation system, SLOs 
focus on using precise evidence regarding growth in student learning in targeted areas to inform major 
changes in instructional practices. They provide a way to measure and improve both teacher practice 
and student academic growth. Overall, SLOs provide a vehicle to ensure that high quality and rigorous 
student assessments are matched by high quality and rigorous instructional practices. 

SLOs are also a new practice for educators in Maryland. While previous chapters explore general issues 
of capacity building, this chapter looks more deeply at specific Issues of implementing SLOs as part of 
TPE. 

Overall Expectations for SLOs 

Most interviewees are receptive to the use of SLOs in teacher evaluation. Across role groups, 
educators like the Idea of using SLOs as a tool to measure student growth and facilitate collaborative 
discussions on instructional strategies and effectiveness. 

Superintendents, for example, see challenges in implementing SLOs well but feel that teachers are 
embracing the SLO process as an Important tool for Improving both student learning and teacher 
practice. Principals agree, though they acknowledge teachers’ learning curve and say that the time 
required remains a concern. Teachers and their union leaders say that SLOs provide a better picture of 
what teachers are doing instructionally, and helps them improve their practice. 


“Time will show that the SLO process has the power to be a better tool for evaluating teachers 
than other approaches." 

-Superintendent 

"Teachers love SLOs because they can see specific growth. The challenge is that the process takes 
so much time. " 

-Principal 

"SLOs are so much better than a test score. They help you look at what you did and what 
strategies you used. They are tied to best practices. This is what excites teachers and helps them 
to get better." 

-Union Leader 


Interviewees recognize many attributes of SLOs as beneficial to assessing teacher performance. 

Teachers and union leaders like the Idea that SLOs yield timely data that they can use to improve 
instruction for students in their classrooms In real time, rather than waiting for end of course or end of 
year assessments. Principals and superintendents see SLOs as an important component of assessing 
teacher performance because the design and evaluation of an SLO focuses on the instructional delivery 
of content to students. 
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"Having conversations on why I'm setting those goals and what instructional strategies to use is 
valuable. We are on the same page. There are richer and deeper conversations with my 
principal." 

-Teacher 

"SLOs provide an opportunity to look at growth measures in ways that give teachers timely 
feedback on how kids are doing and how the teachers can alter their instruction." 

-Union Leader 

"1 told the teachers, 'we are going to use the SLO as a tool to monitor and improve our 
instruction and student performance.' Now we are learning how to use the tool." 

-Principal 

"We are seeing good things happening with SLOs: first, a degree of emphasis on improvement 
and the use of professional learning communities to improve results; second, more attention to 
the alignment of the curriculum; third, more attention to data-driven instruction." 

-Superintendent 


Effect of Prior SLO Experience 

As noted in chapter three, districts and schools with prior TPE training or implementation experience are 
more confident in implementing all aspects of the reform this year. Of particular note is the level of 
working familiarity and facility with SLOs. Over one-half of teachers and principals with prior experience 
feel that teachers receive information about developing high quality SLOs (54%), the use of SLO 
components (54%), guidance on selecting learning content (54%), and the use of pre- and post- 
assessments for SLOs (51 %), compared to about one-third of teachers and principals without prior 
experience receive this information. Those without have prior experience with SLOs struggled during the 
2013-2014 school year. 

"Principals that piloted last year are more comfortable with the process." 

-Union Leader 

"We went out to all the schools to talk about what this would look like. We took our principals 
through a rigorous training. We had a lot of dialogue with MSDE. We didn't leave it solely in 
the lap of the principals. Coordinators work on the SLO development process too." 

-Superintendent 

"We didn't get involved as a system early enough. We're playing catch-up." 

-Superintendent 


The effect of SLO experience in Maryland is consistent with national research from Denver and 
Oharlotte-Mecklenburg (Slotnik and Smith 2004, 2013). It takes time for teachers to become fluent with 
SLOs and for school and district leaders to learn how to support teachers in the SLO process. 
Oharacteristically, it takes two to three years of sustained effort for teachers and principals to develop 
Increasingly higher levels of skill and expertise with implementing high quality SLOs. As with any new 
and meaningful practice, a learning curve is involved at classroom, school and district levels. 
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Experience with this Year’s Implementation 

Interviews and surveys indicate that principals have a more positive impression than teachers about 
how effectively SLOs are being implemented. In particular, principals speak with enthusiasm about 
the coilaborative aspects of SLOs and the value of their SLO-prompted conferences with teachers. By 
contrast, many teachers feel that principals don’t confer with them enough. 

In Figure 8, survey results show that 77% of principals agree that data were used to develop SLOs and 
improve teaching, while 55% of teachers agree. More principals (90%) than teachers (71 %) agree that 
there were opportunities to confer with principals about SLOs. Similarly on the opportunities to engage 
in a mid-year review, 82% of principals agree, compared to 63% of teachers. Finally on whether the 
results of the evaluation would be used to inform professional development plans, 74% of principals 
agree as compared to 45% of teachers. 


Figure 8: Teacher and Principal Views on This Year’s Experience 
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Principals describe their role in conferring with teachers as productive and mutually useful, especially in 
terms of educating teachers about using data in SLOs. Though many educators embrace the idea of 
conducting deeper analysis of student data, some are concerned about whether it may have unintended 
consequences. 


Real Progress in Maryland: Student Learning Objectives and Teacher and Principal Evaluation 


41 




"It's fun to go to those teachers who did not know how to access their own data, pull it up, and 
talk about what they are trying to do with their students. They are so proud of themselves and 
the data they can show me about student progress." 

-Principal 

"Working collaboratively, our teachers are able to see these tools all together. Because of the 
SLOs, our teachers have learned and continue to learn their data better." 

-Principal 


Several teacher interviewees feel they do not have enough input into how SLOs are selected and 
crafted. Schools and districts vary considerably in terms of who actually goes through this thinking 
process and crafts the objectives. Some teachers are given freedom on their classroom SLO, but must 
use a school-wide SLO for their second one. One principal said that teachers welcomed the school-wide 
SLO as a learning opportunity. 


r ^ 

To the extent that schools or 
districts short-circuit the 
teacher thinking process, they 
are in effect undercutting the 
potential benefits of SLOs. 

L J 


"We receive our SLOs. We don't have much say in them. 
We can only adjust the goal or target." 

-Teacher 

"The SLOs are set by the supervisors in our district." 

-Teacher 

"School-wide SLOs may not align with a teacher's 
individual content area." 

-Teacher 


The role of teachers in crafting SLOs is a significant issue. Fifteen years of national practice and research 
make clear that the teacher thinking process is critical to successful SLO implementation and improved 
teacher and student performance. The thinking process includes examining data, identifying key learning 
content and standards, determining appropriate instructional strategies and establishing growth targets. 
These steps all come together with SLOs to help teachers build upon their craft knowledge and 
strengthen the quality of their practice. To the extent that schools or districts short-circuit the teacher 
thinking process, they are in effect undercutting the potential benefits of SLOs. 

The promise of SLOs is that they drive instructional improvement and provide grounded evidence that 
can inform continued professional growth for teachers through targeted professional development. At 
the time of this data collection, districts and schools had not yet produced final teacher ratings, and had 
not finalized how they would use those results in the summer and fall. 

As a result, survey responses show that 30% of teachers do not know whether the results of TPE 
evaluation will be used to inform professional development plans, and 25% expect that they will not. 
Moreover, while 43% of teachers indicate that they are receiving professional development that is 
informed by the SLO process (versus 66% of principals), 37% do not. This suggests that districts either 
do not yet have concrete plans for using TPE results to inform professional development for teachers, or 
they have not yet clearly communicated how they will use the information to their teachers. 
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Table 7: Teacher and Principal Views on Student Learning Objectives 


Student Learning Objectives (SLOs) 

Teachers receive... 

Agree 

Undecided 

Disagree 

Professional development that is informed by the SLO 
process.* 

Principals 

66% 

15% 

20% 

Teachers 

43% 

20% 

37% 


Note: * indicates statistically significant differences at the 95% confidence level. Agree \s a composite of strongly 
agree/agree. Disagree \s a composite of strongly disagree/disagree. 


Challenges of Implementing SLOs 

While interviewees iike the promise of SLOs, many have found the implementation of SLOs in their 
school or district to be challenging for a variety of reasons. Building on the concerns noted in chapter 
four, educators describe four major challenges: 

Challenge Cne: Communicating about SLCs 

Clear and consistent two-way communication is essential when teachers are adopting a new practice 
such as SLOs, just as it is for TPE as a whole. A number of teachers describe the early 
implementation of the SLO process as confusing. Across schools and classrooms, they say that 
information has been inconsistent, conflicting, and changeable— leading to frustration and uneven 
implementation. 


“The expectations for SLOs changed multiple times in the beginning several months." 

-Teacher 

"Preparing the SLO is the biggest difference in practice. The superintendent is saying one thing. 
The principal is saying something else. For teachers, they are unclear on what is right." 

-Union Leader 

“When we had professional development, we were told that teachers would make the decision [on 
setting SLOs], then school principals started to make the decision. That's negative." 

-Teacher 


Challenge Two: Principal Capacity 

Principals have a critical role in the TPE system. As such, the success of the system depends on their 
having the necessary skiils and understandings to implement high quality SLOs (and observations) that 
support the instructional focus of the school. When asked about principal readiness to lead this work, 
interviewees at all levels express reservations about principal readiness to effectively guide the 
SLO process. Among principals and superintendents, many understand that principals need more 
training and they need to develop a deeper understanding of an effective SLO implementation process. 
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"The big anxiety is the SLOs. That piece is still fuzzy, even at my level: What is a rigorous SLO? 
Haw do you tweak things?" 

-Principal 

"Principals are not ready. They are overwhelmed and they are struggling. They struggle as much 
as everyone else. They are not trained well. " 

-Union Leader 

"There needs to be more professional development about how to write SLOs, what do the 
percentages mean, what is a tiered SLO, and how do you re-visit an SLO." 

-Superintendent 


Principals still learning the process can be challenged to find the time required to work with 
teachers individually on their SLOs. Some schools or districts are sharing this responsibility to others, 
such a coordinators, to ease the burden on the principal. 

"It's really very time-consuming when you collect a lot of data. The tools we use have different 
layers, so the time piece is a concern." 

-Principal 

"The workload on the principal as the sole evaluator is enormous. The principals need more 
support from the district." 

-Superintendent 

"Having two SLOs means six meetings with the principal. That's a lot." 

-Teacher 


Challenge Three: Teacher Training and Support 

When asked about the training teachers receive, many interviewees believe that teachers receive 
insufficient training on how to understand and develop an SLO. To implement SLOs well, teachers 
need to understand and master what, for some, are new concepts and skills. The complexity of the task 
requires ongoing professional development that some districts have insufficiently anticipated or 
provided. 

"At the start of the year, administrators trained teachers on SLOs with two hours or less devoted 
to the training. This was our only training on SLOs. We need deeper professional development 
on TPE. Teachers don't understand it yet." 

-Union Leader 

"SLOs are a difficult process. Teachers got very little training. The administrators are new to it." 

-Union Leader 

"The SLOs are well intended but teachers still do not understand them or how to keep data on 
them." 

-Principal 
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Besides training on how to develop an SLO, teacher survey respondents indicate they need to know 
more about the most effective instructional strategies to use to help students meet the SLO targets. 
More teachers disagree (43%) than agree (37%) that they receive guidance on selecting research-based 
instructional strategies for SLOs. In contrast, principals believe (63%) their teachers are receiving this 
guidance. 


Table 8: Teacher and Principal Views on Student Learning Objectives 


Student Learning Objectives (SLOs) 

Teachers receive... 

Agree 

Undecided 

Disagree 

Guidance in selecting research-based instructional 
strategies for SLOs.* 

Principals 

63% 

16% 

22% 

Teachers 

37% 

20% 

43% 


Note: * indicates statistically significant differences at the 95% confidence level. Agree \s a composite of strongly 
agree/agree. Disagree \s a composite of strongly disagree/disagree. 


The research on effective teaching shows that teachers who get better results with students are able to 
link instructionai strategies to the specific content they are helping students to learn. Effective teachers 
understand how to use content specific pedagogy and strategy as well as knowing their content area 
well. In addition, the research on SLO implementation underscores the importance of the Learning 
Content and Instructional Strategies components of an SLO. If a significant percentage of teachers do 
not believe they are receiving guidance in this area, districts need to address this issue directly through 
professional development and instructional support. 

In surveys, principals and teachers identify different views on the support and resources teachers 
receive. Nearly 70% of principals feel that teachers are receiving information about developing high 
quality SLOs and about the use of SLO components. Just as many principals believe teachers get 
resources to support the use of student data for SLOs and guidance on selecting learning content for 
SLOs. Only about one-half of teachers agree that they are receiving information about developing high 
quality SLOs, information about the use of SLO components, and guidance on selecting learning content 
for SLOs. Further, only 43% of teachers agree that they are receiving resources to support the use of 
student data for SLOs. 


Figure 9: Teacher and Principal Views on Student Learning Objectives (SLOs) 
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Principals and teachers also have different views about the support teachers receive in using pre- 
and post-assessments and setting growth targets in SLOs. Nearly 70% of principals believe their 
teachers are receiving recommendations for selecting pre/post assessments for SLOs; 60% believe 
teachers receive valid and reliable approaches to setting growth targets for SLOs. Yet nearly half of 
principals think they as principals need support in the use of pre/post assessments. For teachers, 48% 
believe they were receiving recommendations for selecting pre/post assessments for SLOs and 40% 
indicate they receive valid and reliable approaches to setting growth targets for SLOs. 


"Setting goals for kids is a new way of thinking for many teachers. The biggest struggle is what 
is an effective and fair SLO? What about [factors such as] student attendance that teachers do 
not control." 

-Superintendent 

"Working on SLOs and making sure scores are fair and square is a challenge. It's hard to find 
valid measures for teachers across the board. Trying to fine tune in and find an appropriate 
range that is fair is a challenge." 

-Union Leader 

"Our leadership makes pre-tests, and teachers as groups make the post-tests. When you try to 
line them up, it's messed up. It took us four PD sessions to come up with and figure out the 
tests with the same rigor." 

-Teacher 


One of the strengths of implementing SLOs is that they reveal pre-existing gaps in district capacity that 
need to be addressed. In this instance, SLOs reveal rather than cause gaps in teacher and principal 
facility with assessments and growth target setting. For the dual purposes of supporting effective SLO 
implementation and classroom and school improvement, districts need to consider targeting these two 
areas for further professional development. 


Figure 10: Teacher and Principal Views on Setting SLO Targets/Assessing Performance 
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Challenge Four: Understanding SLO Quality 

High quality SLOs correlate with increases in student academic growth (Slotnik and Smith 2004, 2013). 
This is why an emphasis on quality is core to an effective SLO implementation process. 

Most interviewees grapple with understanding what constitutes a high quality SLO. Many report 
anxiety over whether their SLOs are rigorous, effective, and fair and whether they are “done right.” 

Many educators would like a quality rating rubric to help guide not only the development of SLOs 
but also the review process, where fairness requires inter-rater reliability. Very few interviewees report 
having such a rubric. 


"We would love to see something like a quality rating rubric. Everyone is struggling with the 
question: Is it a good SLO?" 

-Principal 

"Principals would like help with helping teachers make meaningful SLOs. Are the SLOs 
meaningful? Are the SLOs rooted? 1 am not so sure. A quality rating rubric would really help." 

-Principal 

"Our administrators have not been trained well enough to determine the quality of SLOs. This is 
a concern for teachers." 

-Superintendent 


Besides seeking a quality rubric, interviewees speak of wanting to learn more about how to analyze 
strengths and weaknesses of specific SLOs. Interviewees say they would welcome being provided 
with annotated SLOs, which would enable teachers and administrators to examine why each component 
of an SLO, as well as the overall SLO, is an example of varying degrees of quality. 


"We need a treasure trove of vetted, annotated SLOs to look at and learn from." 

-Superintendent 

"We need models of good, bad and ugly SLOs in every subject and grade. The state is moving in 
that direction." 

-Union Leader 


"Annotated SLOs? I wish I had that this year." 


-Principal 


More than half of all principal and teacher survey respondents agree on the need to receive more 
feedback on SLOs. Teachers want feedback from principals; principals seek feedback from district 
administrators. Educators want to better understand SLOs. This level of agreement opens the door for 
meaningful discussions about pedagogy and student progress at the school and district levels. 
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Table 9: Teacher and Principal Views on SLO Feedback 


1 need support in... 

Agree 

Undecided 

Disagree 

Receiving SLO feedback from school or district 
administration. 

Principals 

55% 

14% 

31% 

Teachers 

50% 

19% 

31% 


Note: Agree is a composite of strongly agree/agree. Disagree is a composite of strongly disagree/disagree. 


The interviews and survey responses show the openness of Maryland educators to using SLOs to 
advance student achievement and professional practice— but they want to be supported in doing so. 
Where research indicates effective SLO implementation needs to be and where Maryland educators 
want to be are the same place: a greater understanding in concept and a greater demonstration in 
practice of crafting and implementing high quality SLOs that are supported by the districts’ instructional 
strategy. 
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Chapter Six: Issues and Recommendations 


The following recommendations focus on what the state can directly support, and what district and 
school leaders need to do well, in order to have a high quality implementation of the TPE system in 
general and of the SLO process in particular. They focus on ways to improve the quality and consistency 
of implementation within and across districts in Maryland, while reinforcing the instructional emphasis of 
TPE. 

Being able to successfully manage the TPE system is a core requirement for every district in Maryland. 

In making decisions to advance TPE, implementation needs to be sensitive to important dimensions of 
educator practice. This means that districts must strike a thoughtful balance between manageability and 
having a thorough, rigorous process, and between standardization and customization. This balance is 
essential when implementing SLOs. 

For example, to strengthen both teacher and student performance through an SLO process, districts 
could elect to have teachers craft 4 to 5 pages of narrative explaining their practice in their SLOs— which 
clearly would overburden most teachers. Conversely, to ensure successful completion of the process, a 
district could elect to have teachers engage very little with their SLOs. Where to draw the line on 
balancing manageability and a thorough, rigorous process is important and should include thoughtful 
conversations about what a district needs to emphasize in evaluation and instruction, while 
acknowledging their capacity to successfully manage the TPE process. 

Having comparable standards across schools is extremely important, but not at the cost of losing 
teacher buy-in and the ability to account for important differences in teaching assignments. In making 
decisions on this front, districts would benefit from state-provided guidance on how much flexibility 
schools should be allowed to adapt and contextualize processes, while still having the comparability to 
maintain confidence and trust in the overall TPE system. 

Any of the SLO elements could be standardized across multiple schools in a district. Consider a district 
that decides on the learning content for all second-graders. It could be this standardization helps a 
district advance student learning in key standards that have proven through recent data to be a vital area 
of need across the district. It could also be, however, that deciding on learning content at the district 
level impedes teachers’ abilities to diagnose and address their own students’ needs that may or may not 
match district trend data. 

In striking this balance, districts need to consider how much standardization is enough while allowing for 
customization where necessary and feasible. Moreover, districts would markedly benefit by heeding a 
critical national lesson learned from fifteen years of SLO practice and research: the thinking process 
counts with SLOs. 

The SLO process fosters the planning and delivery of instruction through the use of data and systematic 
analysis. Further, it motivates performance though the setting of growth targets by both teachers and 
students. The process of thinking through the inter-related components of the SLO is what matters 
most. Therefore, it is critical to develop this thinking process rather than distribute “model” SLOs or use 
boilerplate, one-size-fits-all samples (Slotnik and Smith, 2013). 

The recommendations build on the comprehensive support MSDE is currently providing to LEAs. They 
reinforce the Influencing Transformation strategy that guides MSDE’s support to districts. These 
recommendations also should become part of the agenda of the alliance of eight organizations that have 
come together in Maryland to support the effective implementation of SLOs as part of the overall TPE. 
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MSDE has established a strong foundation for implementing TPE. The next phase of implementation 
improvements needs to come at district-to-school and school leader-to-teacher levels. While MSDE will 
continue to provide a range of supports to LEAs, ownership of continuous improvement of TPE needs to 
increase at the local level. 

Issue 1: TPE Integration with Instruction 

Overview 

Maximizing TPE’s instructional focus depends on the ability of districts to integrate the new evaluation 
system within their core instructional frameworks. This means that districts need to be able to implement 
the new evaluation system in ways that provide glue across their instructional initiatives. The national 
experience demonstrates that it takes focused support for new evaluation systems to take root and 
strengthen educator and student performance, while providing a fair and accurate assessment of 
educators’ practice. The analysis of the interviews and survey responses shows that districts are 
interested in and largely committed to strengthening the overall TPE process— including providing 
additional support to help teachers improve instructional practices. 

Recommended Action 

Provide a series of instructionai crosswaik resources to districts. The purpose of these materials is to 
delineate key instructional connections between SLOs and, respectively, teacher observations, 
Maryland’s College and Career-Ready Standards, Maryland’s Instructional Leadership Framework, and 
ISLLC Standards. Implementation is smoother when teachers and principals understand how SLOs 
relate to the other major instructional pieces that are being carried out concurrently. User friendly tools 
that show these connections aid in building such understanding. 

Deveiop protocots to ieverage evidence that can inform professionat devetopment and teadership 
devetopment SLOs and observations provide a rich source of data which can be used to help ascertain 
the current quality of instruction and to identify gaps. The protocols can assist districts to analyze these 
data in order to inform strategies for addressing the professional development needs of teachers and the 
leadership development needs of instructional supervisors and administrators. 

Buitd the capacity of districts to assess and instructionaiiy integrate their TPE impiementation. A way to 
strengthen implementation is to provide districts with templates and training for collecting and analyzing 
TPE data. Districts can then use these templates to examine the degree of rigor and comparability of 
TPE ratings across classroom assignments, grades and schools, in addition, district self-assessment 
protocols can enable districts to assess and benchmark their progress in implementing TPE in the 
context of instructional improvement. 

impact 

By integrating TPE with district instructional frameworks, districts are able to reinforce and support their 
core priorities, while avoiding the silo effect of implementing initiatives in isolation of one another. Doing 
so also helps teachers and principals trust the purpose and results of the evaluation system and know 
how it links to improved instruction and better student results. 

Issue 2: Principal Leadership 

Overview 

Building on MSDE’s new Principals Pipeline initiative, it would be helpful to have a strand that focuses 
on providing targeted supports to current principals in their roles as instructional leaders and stewards 
of the TPE process at the school sites, in particular, the national research from Denver and Charlotte- 
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Mecklenburg highlights the critical role of principals in guiding the effective implementation of SLOs. 
That role is equally important in the conduct of teacher observations. The bottom line is that TPE 
depends on principals who can ensure that high quality, instructional learning is taking place in their 
buildings. 

Recommended Action 

Develop an expanded series of principal leadership prompts. MSDE can provide protocols to assist 
districts and principals to: 

• Facilitate an effective TPE implementation by organizing the school calendar in four phases: 
preparation, development, implementation, and results analysis 

• Lead evidence-based discussions with teachers, drawing on the data from the SLOs and 
teacher observations 

• Use the TPE process to nurture instructional learning via professional learning communities 

Provide a specific stream of support customized to high school principals and their immediate 
supervisors. The interviews and survey responses underscore that the high schools are experiencing 
particular challenges in implementing the new evaluation system. 

Convene a statewide working group of principals. This group can be a model for districts. Its purpose is 
to examine the implementation of TPE within districts and identify areas of strength, weaknesses and 
strategies for promoting more consistent quality at school and district levels. 

impact 

Principals have a particularly critical role in the TPE system. As such, the success of the system 
depends on their having the necessary skills and understandings to implement high quality SLOs and 
observations that support the instructional focus of the school. Developing the skills and understanding 
of principals will take concerted effort over time, and needs to be an ongoing focus of MSDE support to 
districts and district support to schools. 

Issue 3: Quality and Consistency of SLO Implementation 

Overview 

Quality matters when implementing SLOs. Fifteen years of national research and practice emphasize the 
importance of implementing an SLO process at a high level of rigor and quality. Simply put, high quality 
correlates to increases in student academic growth. 

Recommended Action 

Provide districts with a sample SLO Quality Rating Rubric. National practice shows that a 4-level rating 
rubric can be used effectiveiy for the dual purposes of promoting dialogue on instructional planning and 
quality, and ascertaining the quality of individual SLO components and the overall SLO. 

Broaden capacity building to address emerging need areas. As districts become more familiar with the 
nuances and compiexities of implementing SLOs, there is a need for further fieid-tested, practical 
guidance in areas such as target setting, teacher developed assessments and performance-based 
assessments. 
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Provide a resource library of annotated SLOs. Building on steps taken to date, MSDE can promote a 
deeper understanding of quaiity in the crafting of SLOs by providing content for and electronic access to 
annotations of SLOs from diverse grades and subject areas. 

Convene district leaders to examine the requirements of an instructional management system for their 
implementation ofTPE. To support the integration of evaluation and instructional strategies, districts 
need to be able to ensure comparability and rigor across grade levels, subjects, classroom assignments 
and schools. This means being able to nurture and manage a fair and reiiable evaiuation system in reai 
time so that it is a process of continuous improvement in instruction rather than an exercise in 
compliance. This is a significant undertaking in any district and MSDE can facilitate the capacity building 
in this area. 

impact 

High quality SLOs make a difference for students. Research shows that the quality of the SLOs is 
positively and significantly associated with improved student academic growth. District management 
systems need to be structured to advance and support the quality of implementation ofTPE overall and 
SLOs in particular. 

Issue 4: Quality and Consistency of Observations 

Overview 

Principals and teachers need to have a clear and shared understanding of high quality instructional 
practices, how they wili be assessed, and how these practices relate to the observation rubrics and 
processes. This includes having a common understanding of how to taik about instruction, and a 
common understanding of what constitutes a high functioning observation process. A consistent 
observation process needs to include pre- and post-conferences conducted by the observer with the 
teacher. The power of the observation process lies in the dialogue that the teacher and observer have 
about instruction and in the opportunity to model instructional behaviors that can help strengthen any 
areas of weakness. 

Recommended Action 

Provide further resource materials on key aspects of the teacher observation process. The specific 
purpose of the materials is to help deepen the understanding in the schools of the pre-observation 
conference/observation/post-observation conference sequence, the application of the domains, and the 
vaiue of and differences between inter-rater agreement and inter-rater reliabiiity. They should also help 
educators at all levels develop a common language about instruction and effective instructional 
practices. Taking these steps enables the observation process to help drive practice toward higher level 
thinking skiils in aduits— the counterpart to deveioping higher level thinking skiils in students. 

Provide additional professional development toots to help teachers and principals develop better 
understanding of the observation rubrics and the teaching components. These tools can take the form of 
webinars, videos, and regionai training session. As educators gain understanding of the teaching 
components, MSDE and districts then have the opportunity to increase the depth of knowledge of the 
content, and to increase the alignment of the observation process with SLOs, Maryland College and 
Career-Ready Standards, and the new PARCC assessment. 
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Impact 

When districts and schools have a common understanding of what instructional quality means and how 
it will be assessed, and have a rigorous and fair observation process, it facilitates trust and 
instructionally focused dialogue between teachers and supervisors that helps advance student 
outcomes. It is important to conduct an observation process that is fair and comparable across schools 
and classrooms in the district. 

Issue 5: TPE Data Management and Technology Infrastructure 

Overview 

Managing TPE data effectively depends, in part, on having the appropriate technology. In interviews a 
number of districts indicate that they have struggles in this area. Although many districts used their mini- 
grants for district and school hardware/software investments, they now better understand, after a full 
year of implementation, what their data systems can and cannot do. Because districts have varying 
capabilities and face different economies of scale, districts need customized technology solutions to 
manage TPE data in ways that can both inform and strengthen their TPE implementation efforts. 

Recommended Action 

Convene district leaders to identify the criteria for and capabilities of technology solutions to help 
manage the TPE process. Building on MSDE's technical assistance during the past year, the state can 
broaden a multi-district discussion on challenges districts encounter as they implement and use their 
data systems. Together they can develop specifications and technological management requirements of 
a system that facilitates the management of TPE overall and the SLO-related data in particular. Districts 
can then adapt these specifications to their unique needs and fiscal circumstances. One goal of these 
convenings is to continue providing a forum for examining district practices for addressing TPE data 
management issues, allowing other district leaders to assess the benefits and restrictions of those 
practices. 

Impact 

The quality and level of technological support can make or break the implementation of TPE. If flawed, 
frontline educators often blame the reform itself rather than the faultiness of its implementation. This is a 
problem to anticipate and avoid so that the credibility of the new evaluation system is not undercut. 

Issue 6: Constituency Building and Inter-District Learning 

Overview 

Successful implementation of SLOs and the accompanying TPE support system requires clear two-way 
communication and constituency building. Stakeholders in the districts must not only understand the 
new evaluation system but have the opportunity to contribute to refining and improving its 
implementation. This type of feedback requires strategic communication and systematic engagement of 
educators from the school to the district, and back again. When a major systemic initiative such as TPE 
is introduced, the forces of misinformation are often more powerful than the forces of accurate 
information. This is why having a focus on two-way communication and engagement is a cornerstone of 
effective implementation. 

Recommended Action 

Convene a working group on two-way communications. This should be a select group of district and 
school level practitioners respected for their communication approaches. Their focus should be to 
identify field proven, effective ways for advancing two-way, consistent communications within and 
between districts and schools. 


Real Progress in Maryland: Student Learning Objectives and Teacher and Principal Evaluation 


53 


Develop an interactive web site for sharing vetted practices. There is interest from frontline educators in 
learning about promising practices in TPE implementation from other districts. MSDE can help address 
this need by providing an electronic site where promising practices, vetted by MSDE’s TPE team, can be 
shared on an inter-district basis. 

There is a great deal districts and schools can learn by sharing their experiences with one another. At 
the school level, principals and teachers could learn from emerging practices that are making a 
difference. At the LEA level, a district could compare the Intensity of its evaluator training with similar 
districts. If a district’s summative ratings are significantly different across schools, they could contact 
leaders of a similar district with more consistent ratings to better understand how they train their 
evaluators. 

impact 

Implementation is stronger when there are informed opportunities for people undergoing the same 
change process to learn from one other. MSDE is uniquely positioned to encourage and increase such 
learnings. In June 2014, the state's leading education organizations— the Maryland State Board of 
Education, Maryland State Department of Education, Maryland State Educators Association, Public 
School Superintendents Association of Maryland, Maryland Association of Boards of Education, 
Maryland Association of Secondary School Principals, Maryland Association of Elementary School 
Principals, and the Baltimore Teachers Union— agreed to coordinate resources and strategies in support 
of rigorous and measurable SLOs. This partnership gives Maryland a unique platform to inform and learn 
from constituents across the state. 

Summary 

MSDE Is making significant strides In guiding and supporting the Implementation of SLOs as well as TPE 
overall. The more experience frontline educators have with the new evaluation system, the higher their 
skill and comfort levels are with its implementation and the more their efforts focus on strengthening 
instruction. The dual challenge of improving the quality and consistency of implementation within and 
across districts is not unusual In a major new state undertaking. With the ongoing high quality support 
from the state, districts can directly address this challenge through thoughtful and coordinated action in 
six key issue areas. Maryland’s landmark alliance of eight pivotal organizations further strengthens the 
foundation and prospects for continued progress in the year ahead. 


Slotnik, W., Smith, M. etal. (2004). Catalyst for change: Pay for performance in Denver. Boston, MA: Community Training and 
Assistance Center. 

Siotnik, W., Smith, M. etal. (2013). It’s more than money: Teacher incentive fund-leadership for educators’ advanced performance, 
Chariotte-Mecktenburg schools. Boston, MA: Community Training and Assistance Center. 
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Addendum 


Figure 1: State Teacher and Principal Models 
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Source: The Maryland Teacher and Principal Evaluation Guidebook, Version 3, http://msde. state. md.us/tpe/TPE Guidance Versions 092013.pdf 
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Figure 2: Local Teacher and Principal Models 
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Table 1: Survey Response Rates by District 


District 

Response Rate 

Ailegany County 

31 .3% 

Anne Arundei County 

26.8% 

Baltimore City 

5.7% 

Baltimore County 

23.7% 

Calvert County 

26.7% 

Caroline County 

29.6% 

Carroll County 

43.7% 

Cecil County 

72.6% 

Charles County 

36.5% 

Dorchester County 

43.2% 

Frederick County 

26.6% 

Garrett County 

57.6% 

Harford County 

49.0% 

Howard County 

51 .9% 

Kent County 

59.6% 

Montgomery County 

0.1% 

Prince George's County 

26.6% 

Cueen Anne's County 

55.3% 

Somerset County 

62.7% 

St. Mary's County 

25.1% 

Talbot County 

51 .6% 

Washington County 

52.4% 

Wicomico County 

29.0% 

Worcester County 

40.5% 


Note: Of the 23 districts that responded to the survey, the response rate is 31 .3%. 
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Table 2: Characteristics of the Survey Respondents (n=16,314) 



n 

% 

I have*: 

Participated in classroom observation training(s) 

7,201 

44% 

Participated in Student Learning Objectives (SLO) training 

14,039 

86% 

Accessed other source(s) of information or experiences related to TPE 

4,204 

26% 

Accessed other source(s) of information or experiences related to SLOs 

8,031 

49% 

The school level I work in is: 

Elementary school 

7,777 

48% 

Middle school 

3,759 

23% 

High school 

4,387 

27% 

Not Applicable / No Response 

391 

2% 

My position is described by this educator category: 

Classroom teacher 

11,108 

68% 

Non-classroom teaching staff 

1,858 

11% 

Special educator 

1,939 

12% 

Principal 

622 

4% 

Assistant principal 

407 

2% 

Other administrator 

92 

1% 

Central office staff 

149 

1% 

No Response 

139 

1% 

In my position, I work in the following subject area(s)*: 

Arts 

1,368 

8% 

Career and Technical Education 

904 

6% 

Elementary (multiple subject areas) 

5,021 

31% 

English/Language Arts 

3,360 

21% 

English as a Second Language 

435 

3% 

Health and Physical Education 

1,020 

6% 

Math 

2,917 

18% 

Science 

2,376 

15% 

Social Studies 

2,241 

14% 

Special Education 

2,090 

13% 

Worid Languages 

633 

4% 

Other (e.g., music) 

1,308 

8% 

My years of teaching experience are (teachers only; n=1 4,905): 

0-3 

1,615 

11% 

4-6 

1,542 

10% 

7-9 

2,170 

15% 

10+ 

9,422 

63% 

No Response 

156 

1% 

My years of experience as a principai are (principais oniy; n=1 ,029): 

0-3 

282 

27% 

4-6 

170 

17% 

7-9 

187 

18% 

10+ 

261 

25% 

No Response 

129 

13% 

The highest educational degree 1 have attained is: 

Bachelor's degree 

3,419 

21% 

Master's degree 

12,346 

76% 

Doctoral degree 

344 

2% 

No Response 

205 

1% 


‘Denotes multiple choice survey items; percentages do not add up to 1 00. 
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Table 3. Responses from Principals and Teachers (n=1 5,934) 



Principals 

(n=1,029) 

Teachers 
(n=1 4,905) 

Total 

Agree 

Undecided 

Total 

Disagree 

Total 

Agree 

Undecided 

Total 

Disagree 

Systemic Context 

The expectations are clear for teachers and principals in the 
TPE system. 

70% 

13% 

17% 

50% 

20% 

30% 

TPE is based on principles of continuous improvement. 

86% 

9% 

6% 

60% 

23% 

17% 

TPE recognizes the scope of an educator’s roles and 
responsibilities. 

71% 

14% 

16% 

46% 

21% 

33% 

The State's support of the implementation of the TPE 
system is helpful. 

44% 

35% 

21% 

25% 

39% 

37% 

My district’s support of the implementation of the TPE 
system is helpful. 

77% 

13% 

10% 

40% 

31% 

29% 

My school’s support of the implementation of the TPE 
system is helpful. 

88% 

9% 

3% 

55% 

23% 

21% 

The school community’s support of the implementation of 
the TPE system is helpful. 

45% 

41% 

14% 

27% 

41% 

32% 

Quality of TPE Frameworks and Processes 

Teacher evaluation frameworks and processes... 

Respect educators’ professional knowledge and skills. 

85% 

7% 

8% 

53% 

17% 

30% 

Utilize validated observation measures/instruments. 

76% 

14% 

10% 

44% 

23% 

33% 

Are conducted by observers/evaluators qualified to do the 
evaluation. 

90% 

6% 

4% 

61% 

19% 

20% 

Provide a summative rating of educator performance. 

84% 

9% 

7% 

51% 

23% 

27% 

Provide useful feedback to teachers. 

84% 

11% 

5% 

49% 

24% 

27% 

Encourage reflection on instructional practices. 

89% 

7% 

4% 

66% 

17% 

17% 

Lead to improved decisions related to instructional 
approaches. 

79% 

15% 

6% 

48% 

26% 

26% 

Student Learning Objectives (SLOs) 

Teachers receive... 

Information about developing high quality SLOs. 

70% 

13% 

17% 

50% 

16% 

34% 

Information about the use of SLO components. 

74% 

11% 

15% 

50% 

17% 

33% 

Resources supporting the use of student baseline data. 

72% 

11% 

18% 

43% 

19% 

37% 

Guidance in selecting learning content for SLOs. 

76% 

10% 

15% 

50% 

16% 

34% 

Guidance in selecting research-based instructional 
strategies for SLOs. 

63% 

16% 

22% 

37% 

20% 

43% 

Information on connecting Maryland's College and Career- 
Ready Standards to SLOs. 

55% 

19% 

26% 

33% 

23% 

43% 

Recommendations for selecting pre/post assessments for 
SLOs. 

71% 

12% 

18% 

48% 

16% 

36% 

Valid and reliable approaches to setting growth targets for 
SLOs. 

60% 

15% 

25% 

40% 

20% 

41% 
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Principals 

(n=1,029) 

Teachers 
(n=1 4,905) 

Total 

Agree 

Undecided 

Total 

Disagree 

Total 

Agree 

Undecided 

Total 

Disagree 

Professional development that is informed by the SLO 
process. 

66% 

15% 

20% 

43% 

20% 

37% 

Guidance in developing strategies for student differentiation 
or accommodation where appropriate. 

64% 

14% 

22% 

37% 

20% 

42% 

This Year’s Experience 

This year, 1 believe... 

Data were used to develop my SLOs and to improve my 
teaching. 

77% 

14% 

9% 

55% 

17% 

28% 

There were opportunities to confer with my principal about 
my SLOs. 

90% 

5% 

5% 

71% 

9% 

20% 

There were opportunities to engage in a mid-interval review 
and refinement of my SLOs. 

82% 

7% 

11% 

63% 

11% 

26% 

Reflection on my instructional practices deepened. 

72% 

19% 

9% 

44% 

21% 

36% 

There was more instructionally focused dialogue with my 
colleagues. 

75% 

16% 

9% 

42% 

18% 

40% 

There was more instructionally focused dialogue with my 
supervisor. 

78% 

13% 

9% 

39% 

19% 

42% 

The results of my evaluation will be used to inform my 
professional development plan for next year. 

74% 

20% 

6% 

45% 

30% 

25% 

Capacity Building 

1 need support in... 

Gaining timely access to student data. 

55% 

9% 

35% 

47% 

15% 

38% 

Analyzing student data for action. 

55% 

7% 

38% 

46% 

14% 

40% 

Using pre/post assessments. 

49% 

10% 

42% 

38% 

14% 

48% 

Using Maryland's Gollege and Career Ready Standards. 

63% 

11% 

25% 

55% 

19% 

26% 

Receiving SLO feedback from school or district 
administration. 

55% 

14% 

31% 

50% 

19% 

31% 

Using data from SLOs and observations to strengthen 
school improvement planning. 

64% 

11% 

25% 

48% 

22% 

30% 

Using data from SLOs and observations to make 
improvements in my instructional practices (for teachers 
only). 

N/A 

N/A 

N/A 

48% 

18% 

34% 

Using data from SLOs and observations to make 
improvements in my instructional supervision (for principals 
only). 

70% 

9% 

21% 

N/A 

N/A 

N/A 


Note: Principals reiers to principals and assistant principals; Teachers reiers to classroom teachers, non-classroom teaching staff, 
and special educators. Total Agree is a composite of strongly agree/agree and Total Disagree is a composite of strongly 
d isagree/d isag ree. 
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Table 4. Responses by Participants’ Experience with TPE and SLOs (n=16,314) 



Participated in 
Training(s) on Ciassroom 
Observation and/or SLOs 

Accessed 

Information Reiated 
to TPE and/or SLOs 

Neither Participated 
in Trainings nor Accessed 
Information 

Total 

Agree 

Undecided 

Total 

Disagree 

Total 

Agree 

Undecided 

Total 

Disagree 

Total 

Agree 

Undecided 

Total 

Disagree 

Systemic Context 

The expectations are clear for teachers and 
principals in the TPE system. 

53% 

19% 

28% 

41% 

23% 

36% 

42% 

24% 

34% 

TPE is based on principles of continuous 
improvement. 

63% 

21% 

16% 

55% 

28% 

18% 

50% 

29% 

21% 

TPE recognizes the scope of an educator’s 
roles and responsibilities. 

48% 

20% 

31% 

40% 

25% 

35% 

40% 

22% 

38% 

The State's support of the implementation 
of the TPE system is helpful. 

27% 

39% 

35% 

20% 

39% 

40% 

22% 

39% 

39% 

My district’s support of the implementation 
of the TPE system is helpful. 

44% 

29% 

27% 

33% 

32% 

36% 

32% 

32% 

36% 

My school’s support of the implementation 
of the TPE system is helpful. 

59% 

22% 

19% 

45% 

28% 

27% 

43% 

24% 

32% 

The school community’s support of the 
implementation of the TPE system is 
helpful. 

29% 

42% 

30% 

21% 

42% 

37% 

25% 

40% 

36% 

Quality of TPE Frameworks and Processes 

Teacher evaluation frameworks and processes... 

Respect educators’ professional 
knowledge and skills. 

57% 

16% 

28% 

48% 

20% 

32% 

47% 

20% 

33% 

Utilize validated observation 
measures/instruments. 

47% 

22% 

30% 

40% 

24% 

35% 

39% 

26% 

35% 

Are conducted by observers/evaluators 
qualified to do the evaluation. 

64% 

17% 

18% 

56% 

23% 

21% 

57% 

20% 

23% 

Provide a summative rating of educator 
performance. 

54% 

21% 

25% 

45% 

27% 

28% 

46% 

26% 

28% 

Provide useful feedback to teachers. 

52% 

23% 

25% 

45% 

26% 

29% 

43% 

28% 

29% 

Encourage reflection on instructional 
practices. 

69% 

16% 

16% 

62% 

19% 

19% 

57% 

22% 

21% 

Lead to improved decisions related to 
instructional approaches. 

51% 

25% 

24% 

45% 

29% 

26% 

43% 

29% 

28% 

Student Learning Objectives (SLOs) 

Teachers receive... 

Information about developing high quality 
SLOs. 

54% 

15% 

31% 

32% 

20% 

48% 

33% 

18% 

49% 

Information about the use of SLO 
components. 

54% 

16% 

30% 

35% 

18% 

46% 

34% 

19% 

48% 

Resources supporting the use of student 
baseline data. 

47% 

18% 

35% 

33% 

22% 

44% 

31% 

22% 

47% 

Guidance in selecting learning content for 
SLOs. 

54% 

15% 

31% 

38% 

19% 

43% 

37% 

18% 

46% 

Guidance in selecting research-based 
instructional strategies for SLOs. 

41% 

19% 

40% 

28% 

21% 

51% 

28% 

21% 

51% 

Information on connecting Maryland's 
Oollege and Career-Ready Standards to 
SLOs. 

36% 

23% 

41% 

25% 

24% 

51% 

24% 

27% 

50% 

Recommendations for selecting pre/post 
assessments for SLOs. 

51% 

16% 

34% 

38% 

19% 

43% 

34% 

22% 

43% 
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Participated in 
Training(s) on Classroom 
Observation and/or SLOs 

Accessed 

Information Related 
to TPE and/or SLOs 

Neither Participated 
in Trainings nor Accessed 
Information 

Total 

Agree 

Undecided 

Total 

Disagree 

Total 

Agree 

Undecided 

Total 

Disagree 

Total 

Agree 

Undecided 

Total 

Disagree 

Valid and reliable approaches to setting 
growth targets for SLOs. 

43% 

19% 

38% 

29% 

22% 

48% 

30% 

21% 

49% 

Professional development that is informed 
by the SLO process. 

47% 

19% 

34% 

30% 

22% 

48% 

27% 

22% 

51% 

Guidance in developing strategies for 
student differentiation or accommodation 
where appropriate. 

41% 

20% 

40% 

28% 

23% 

49% 

27% 

22% 

50% 

This Year’s Experience 

This year, 1 believe... 

Data were used to develop my SLOs and 
to improve my teaching. 

58% 

17% 

26% 

45% 

24% 

31% 

41% 

23% 

36% 

There were opportunities to confer with my 
principal about my SLOs. 

73% 

9% 

18% 

60% 

15% 

25% 

58% 

16% 

26% 

There were opportunities to engage in a 
mid-interval review and refinement of my 
SLOs. 

66% 

11% 

24% 

48% 

17% 

35% 

49% 

17% 

34% 

Reflection on my instructional practices 
deepened. 

46% 

20% 

33% 

38% 

26% 

36% 

37% 

23% 

40% 

There was more instructionally focused 
dialogue with my colleagues. 

45% 

18% 

37% 

39% 

20% 

41% 

34% 

23% 

42% 

There was more instructionally focused 
dialogue with my supervisor. 

43% 

18% 

39% 

33% 

23% 

45% 

32% 

22% 

46% 

The results of my evaluation will be used to 
inform my professional development plan 
for next year. 

48% 

29% 

23% 

41% 

34% 

25% 

35% 

35% 

30% 

Capacity Building 

1 need support in... 

Gaining timely access to student data. 

48% 

14% 

38% 

50% 

17% 

33% 

45% 

18% 

36% 

Analyzing student data for action. 

46% 

13% 

41% 

50% 

16% 

35% 

47% 

17% 

36% 

Using pre/post assessments. 

39% 

13% 

48% 

40% 

16% 

44% 

38% 

20% 

42% 

Using Maryland's College and Career 
Ready Standards. 

56% 

19% 

26% 

57% 

22% 

21% 

56% 

22% 

22% 

Receiving SLO feedback from school or 
district administration. 

50% 

19% 

32% 

54% 

22% 

23% 

52% 

20% 

27% 

Using data from SLOs and observations to 
strengthen school Improvement planning. 

49% 

21% 

30% 

52% 

25% 

23% 

49% 

24% 

27% 

Using data from SLOs and observations to 
make improvements in my instructional 
practices (for teachers only). 

48% 

17% 

35% 

51% 

21% 

28% 

52% 

18% 

30% 

Using data from SLOs and observations to 
make improvements in my instructional 
supervision (for principals only). 

70% 

9% 

21% 

50% 

0% 

50% 

82% 

0% 

18% 


Note: Total Agree is a composite of strongly agree/agree and Total Disagree is a composite of strongly disagree/disagree. 
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Table 5. Responses from Principals anc 


Teachers to the Same Questions in 2013 and 2014 



Year 

Principals 

Teachers 

Total 

Agree 

Undecided 

Total 

Disagree 

Total 

Agree 

Undecided 

Total 

Disagree 

Systemic Context 

The expectations are clear for teachers and 
principals in the TPE system. 

2013 

55% 

26% 

20% 

41% 

31% 

28% 

2014 

70% 

13% 

17% 

50% 

20% 

30% 

TPE is based on principles of continuous 
improvement. 

2013 

83% 

12% 

5% 

58% 

29% 

13% 

2014 

86% 

9% 

6% 

60% 

23% 

17% 

TPE recognizes the scope of an educator’s roles 
and responsibilities. 

2013 

67% 

16% 

16% 

42% 

31% 

27% 

2014 

71% 

14% 

16% 

46% 

21% 

33% 

Quaiity of TPE Frameworks and Processes 

Teacher evaluation frameworks and processes... 

Respect educators’ professional knowledge and 
skills. 

2013 

82% 

11% 

8% 

51% 

25% 

24% 

2014 

85% 

7% 

8% 

53% 

17% 

30% 

Utilize validated observation 
measures/instruments. 

2013 

67% 

25% 

8% 

40% 

33% 

27% 

2014 

76% 

14% 

10% 

44% 

23% 

33% 

Provide a summative rating of educator 
performance. 

2013 

71% 

20% 

9% 

42% 

33% 

25% 

2014 

84% 

9% 

7% 

51% 

23% 

27% 

Lead to improved decisions related to instructional 
approaches. 

2013 

66% 

24% 

10% 

38% 

38% 

25% 

2014 

79% 

15% 

6% 

48% 

26% 

26% 

Student Learning Objectives (SLOs) 

Teachers receive... 

Information about developing high quality SLOs. 

2013 

57% 

20% 

23% 

42% 

26% 

32% 

2014 

70% 

13% 

17% 

50% 

16% 

34% 

Information about the use of SLO components. 

2013 

64% 

22% 

14% 

44% 

27% 

30% 

2014 

74% 

11% 

15% 

50% 

17% 

33% 

Resources supporting the use of student baseline 
data. 

2013 

56% 

22% 

22% 

35% 

31% 

34% 

2014 

72% 

11% 

18% 

43% 

19% 

37% 

Guidance in selecting learning content for SLOs. 

2013 

50% 

26% 

25% 

32% 

32% 

35% 

2014 

76% 

10% 

15% 

50% 

16% 

34% 
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Year 

Principals 

Teachers 

Total 

Agree 

Undecided 

Total 

Disagree 

Total 

Agree 

Undecided 

Total 

Disagree 

Guidance in selecting research-based instructional 
strategies for SLOs. 

2013 

45% 

28% 

27% 

27% 

35% 

38% 

2014 

63% 

16% 

22% 

37% 

20% 

43% 

Recommendations for selecting pre/post 
assessments for SLOs. 

2013 

44% 

27% 

29% 

29% 

33% 

37% 

2014 

71% 

12% 

18% 

48% 

16% 

36% 

Valid and reliable approaches to setting growth 
targets for SLOs. 

2013 

42% 

31% 

27% 

28% 

36% 

37% 

2014 

60% 

15% 

25% 

40% 

20% 

41% 

Capacity Building 

1 need support in... 

Gaining timely access to student data. 

2013 

51% 

12% 

37% 

55% 

16% 

28% 

2014 

55% 

9% 

35% 

47% 

15% 

38% 

Analyzing student data for action. 

2013 

44% 

11% 

45% 

53% 

15% 

32% 

2014 

55% 

7% 

38% 

46% 

14% 

40% 

Using pre/post assessments. 

2013 

48% 

13% 

39% 

48% 

16% 

36% 

2014 

49% 

10% 

42% 

38% 

14% 

48% 

Receiving SLO feedback from school or district 
administration. 

2013 

67% 

15% 

18% 

66% 

21% 

12% 

2014 

55% 

14% 

31% 

50% 

19% 

31% 


Note: Principals reiers to principals and assistant principals; Teachers reiers to classroom teachers, non-classroom teaching staff, 
and special educators. Total Agree is a composite of strongly agree/agree and Total Disagree is a composite of strongly 
disagree/disagree. N (principal, 2013)=173; N (principal, 2014)=1,029; N (teacher, 2013)=1,657; N (teacher, 201 4)=1 4,905. 
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Table 6. Teachers’ Responses by Years of Teaching Experience (n=1 4,905) 



0-3 Years 

4-6 Years 

7-9 Years 

10+ Years 

A 

U 

D 

A 

U 

D 

A 

U 

D 

A 

U 

D 

Systemic Context 

The expectations are clear for teachers 
and principals in the TPE system. 

60% 

18% 

22% 

51% 

18% 

31% 

51% 

19% 

29% 

48% 

20% 

32% 

TPE is based on principles of 
continuous improvement. 

68% 

20% 

12% 

60% 

24% 

16% 

60% 

23% 

17% 

59% 

24% 

18% 

TPE recognizes the scope of an 
educator’s roles and responsibilities. 

56% 

21% 

23% 

48% 

21% 

32% 

46% 

21% 

33% 

43% 

21% 

35% 

The State's support of the 
implementation of the TPE system is 
heipful. 

33% 

40% 

27% 

25% 

37% 

38% 

24% 

39% 

37% 

23% 

39% 

38% 

My district’s support of the 
implementation of the TPE system is 
heipful. 

46% 

31% 

24% 

39% 

30% 

31% 

40% 

31% 

29% 

39% 

31% 

30% 

My school’s support of the 
implementation of the TPE system is 
helpful. 

59% 

23% 

18% 

56% 

23% 

21% 

56% 

23% 

22% 

54% 

24% 

22% 

The school community’s support of the 
implementation of the TPE system is 
helpful. 

36% 

40% 

24% 

27% 

38% 

34% 

27% 

41% 

32% 

25% 

42% 

33% 

Quality of TPE Frameworks and Processes 

Teacher evaluation frameworks and processes... 

Respect educators’ professional 
knowledge and skills. 

67% 

14% 

18% 

56% 

16% 

28% 

52% 

17% 

31% 

51% 

17% 

32% 

Utilize validated observation 
measures/instruments. 

58% 

19% 

23% 

46% 

21% 

33% 

44% 

23% 

33% 

41% 

24% 

34% 

Are conducted by observers/evaluators 
qualified to do the evaluation. 

75% 

12% 

13% 

66% 

16% 

18% 

65% 

17% 

19% 

58% 

21% 

21% 

Provide a summative rating of educator 
performance. 

62% 

18% 

20% 

51% 

22% 

27% 

51% 

21% 

29% 

48% 

24% 

28% 

Provide useful feedback to teachers. 

64% 

18% 

18% 

51% 

21% 

28% 

50% 

24% 

27% 

46% 

26% 

28% 

Encourage reflection on instructional 
practices. 

77% 

13% 

10% 

69% 

15% 

16% 

66% 

17% 

17% 

64% 

18% 

19% 

Lead to improved decisions related to 
instructional approaches. 

63% 

20% 

17% 

51% 

24% 

25% 

49% 

26% 

24% 

45% 

28% 

28% 

Student Learning Objectives (SLOs) 

Teachers receive... 

Information about developing high 
quality SLOs. 

47% 

16% 

37% 

47% 

14% 

38% 

48% 

16% 

36% 

51% 

15% 

33% 

Information about the use of SLO 
components. 

47% 

17% 

36% 

48% 

15% 

37% 

49% 

16% 

35% 

51% 

17% 

32% 

Resources supporting the use of 
student baseline data. 

43% 

20% 

37% 

40% 

18% 

42% 

43% 

18% 

39% 

44% 

20% 

36% 

Guidance in selecting learning content 
for SLOs. 

50% 

16% 

34% 

48% 

15% 

38% 

49% 

16% 

35% 

51% 

15% 

33% 

Guidance in selecting research-based 
instructional strategies for SLOs. 

38% 

20% 

42% 

34% 

18% 

47% 

36% 

19% 

45% 

38% 

20% 

42% 

Information on connecting Maryland's 
College and Career-Ready Standards to 
SLOs. 

35% 

23% 

42% 

32% 

20% 

48% 

33% 

22% 

46% 

34% 

24% 

42% 
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0-3 Years 

4-6 Years 

7-9 Years 

10+ Years 

A 

U 

D 

A 

U 

D 

A 

U 

D 

A 

U 

D 

Recommendations for selecting 
pre/post assessments for SLOs. 

48% 

16% 

36% 

47% 

15% 

39% 

48% 

16% 

36% 

48% 

17% 

36% 

Valid and reliable approaches to setting 
growth targets for SLOs. 

43% 

19% 

38% 

39% 

18% 

43% 

39% 

19% 

42% 

39% 

20% 

40% 

Professionai development that is 
informed by the SLO process. 

47% 

18% 

34% 

43% 

19% 

38% 

43% 

18% 

38% 

43% 

20% 

37% 

Guidance in developing strategies for 
student differentiation or 
accommodation where appropriate. 

41% 

20% 

40% 

37% 

18% 

45% 

36% 

19% 

45% 

37% 

21% 

42% 

This Year’s Experience 

This year, 1 believe... 

Data were used to develop my SLOs 
and to improve my teaching. 

58% 

19% 

23% 

53% 

16% 

31% 

54% 

17% 

29% 

55% 

17% 

28% 

There were opportunities to confer with 
my principal about my SLOs. 

70% 

10% 

20% 

69% 

9% 

22% 

70% 

11% 

19% 

72% 

9% 

19% 

There were opportunities to engage in a 
mid-interval review and refinement of 
my SLOs. 

63% 

11% 

25% 

60% 

10% 

30% 

61% 

12% 

27% 

63% 

11% 

25% 

Reflection on my instructional practices 
deepened. 

53% 

20% 

27% 

42% 

21% 

37% 

41% 

21% 

38% 

43% 

21% 

37% 

There was more instructionally focused 
dialogue with my colleagues. 

47% 

22% 

32% 

39% 

17% 

44% 

40% 

17% 

43% 

42% 

18% 

40% 

There was more instructionally focused 
dialogue with my supervisor. 

48% 

21% 

32% 

37% 

20% 

43% 

39% 

18% 

43% 

39% 

18% 

43% 

The results of my evaluation will be used 
to inform my professional development 
plan for next year. 

56% 

24% 

19% 

46% 

28% 

26% 

43% 

29% 

28% 

43% 

32% 

25% 

Capacity Buiiding 

1 need support in... 

Gaining timely access to student data. 

48% 

18% 

35% 

49% 

13% 

38% 

46% 

14% 

39% 

47% 

15% 

38% 

Analyzing student data for action. 

51% 

15% 

34% 

46% 

12% 

42% 

43% 

13% 

44% 

46% 

14% 

40% 

Using pre/post assessments. 

43% 

15% 

42% 

35% 

13% 

52% 

35% 

13% 

52% 

38% 

13% 

48% 

Using Maryland's Gollege and Career 
Ready Standards. 

57% 

19% 

24% 

59% 

16% 

25% 

55% 

18% 

28% 

55% 

20% 

25% 

Receiving SLO feedback from school or 
district administration. 

51% 

19% 

30% 

54% 

15% 

30% 

50% 

18% 

32% 

49% 

20% 

31% 

Using data from SLOs and observations 
to strengthen school improvement 
planning. 

50% 

24% 

26% 

52% 

18% 

30% 

48% 

20% 

32% 

47% 

23% 

30% 

Using data from SLOs and observations 
to make improvements in my 
instructional practices (for teachers 
only). 

55% 

16% 

29% 

51% 

15% 

33% 

46% 

17% 

36% 

47% 

18% 

35% 

Using data from SLOs and observations 
to make improvements in my 
instructional supervision (for principals 
only). 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 


Note: D = Total Disagree, a composite of strongly disagree/disagree. U = Undecided. A = Total Agree, a composite of strongly 
agree/agree. 
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Table 7. Teachers’ Responses by School Level (n=14,905) 



Elementary School 

Middle School 

High Schooi 

A 

U 

D 

A 

U 

D 

A 

U 

D 

Systemic Context 

The expectations are clear for teachers and 
principals in the TPE system. 

54% 

20% 

26% 

48% 

21% 

32% 

45% 

19% 

37% 

TPE is based on principles of continuous 
improvement. 

66% 

22% 

12% 

59% 

23% 

18% 

52% 

25% 

24% 

TPE recognizes the scope of an educator’s roles 
and responsibilities. 

50% 

21% 

29% 

45% 

21% 

34% 

39% 

22% 

39% 

The State's support of the implementation of the 
TPE system is helpful. 

28% 

40% 

32% 

22% 

40% 

38% 

21% 

36% 

43% 

My district’s support of the implementation of 
the TPE system is helpful. 

44% 

30% 

26% 

37% 

33% 

30% 

36% 

31% 

33% 

My school’s support of the implementation of the 
TPE system is helpful. 

60% 

22% 

18% 

52% 

25% 

23% 

50% 

24% 

25% 

The school community’s support of the 
implementation of the TPE system is helpful. 

30% 

42% 

28% 

25% 

41% 

34% 

23% 

40% 

37% 

Quality of TPE Frameworks and Processes 

Teacher evaluation frameworks and processes... 

Respect educators’ professional knowledge and 
skills. 

59% 

16% 

25% 

51% 

17% 

32% 

45% 

17% 

38% 

Utilize validated observation 
measures/instruments. 

50% 

23% 

27% 

41% 

23% 

35% 

37% 

23% 

40% 

Are conducted by observers/evaluators qualified 
to do the evaluation. 

70% 

16% 

14% 

59% 

20% 

22% 

50% 

22% 

28% 

Provide a summative rating of educator 
performance. 

56% 

22% 

22% 

48% 

23% 

28% 

43% 

23% 

34% 

Provide usefui feedback to teachers. 

55% 

24% 

21% 

47% 

24% 

29% 

41% 

25% 

35% 

Encourage reflection on instructional practices. 

71% 

16% 

13% 

64% 

18% 

18% 

59% 

18% 

23% 

Lead to improved decisions related to 
instructional approaches. 

55% 

25% 

20% 

45% 

28% 

27% 

39% 

27% 

34% 

Student Learning Objectives (SLOs) 

Teachers receive... 

Information about developing high quality SLOs. 

53% 

15% 

32% 

49% 

16% 

35% 

46% 

16% 

38% 

Information about the use of SLO components. 

53% 

17% 

31% 

49% 

17% 

34% 

46% 

16% 

38% 

Resources supporting the use of student 
baseline data. 

47% 

20% 

33% 

43% 

18% 

38% 

38% 

19% 

44% 

Guidance in selecting learning content for SLOs. 

54% 

15% 

31% 

50% 

16% 

35% 

44% 

16% 

40% 

Guidance in selecting research-based 
instructional strategies for SLOs. 

41% 

20% 

39% 

36% 

20% 

44% 

32% 

20% 

48% 

Information on connecting Maryland's College 
and Career-Ready Standards to SLOs. 

37% 

24% 

38% 

31% 

22% 

47% 

29% 

21% 

50% 

Recommendations for selecting pre/post 
assessments for SLOs. 

51% 

16% 

33% 

48% 

15% 

37% 

42% 

16% 

42% 

Valid and reliable approaches to setting growth 
targets for SLOs. 

44% 

20% 

36% 

38% 

19% 

42% 

34% 

19% 

47% 
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Elementary School 

Middle School 

High School 

A 

U 

D 

A 

U 

D 

A 

U 

D 

Professional development that is informed by the 
SLO process. 

45% 

20% 

35% 

44% 

20% 

36% 

41% 

19% 

40% 

Guidance in developing strategies for student 
differentiation or accommodation where 
appropriate. 

41% 

21% 

39% 

36% 

20% 

44% 

33% 

20% 

48% 

This Year’s Experience 

This year, 1 believe... 

Data were used to develop my SLOs and to 
improve my teaching. 

63% 

16% 

22% 

52% 

19% 

29% 

43% 

19% 

37% 

There were opportunities to confer with my 
principal about my SLOs. 

76% 

9% 

16% 

70% 

10% 

20% 

63% 

10% 

27% 

There were opportunities to engage in a mid- 
interval review and refinement of my SLOs. 

65% 

11% 

23% 

62% 

11% 

27% 

58% 

12% 

30% 

Reflection on my instructional practices 
deepened. 

49% 

21% 

31% 

42% 

21% 

37% 

36% 

20% 

44% 

There was more instructionally focused dialogue 
with my colleagues. 

48% 

18% 

34% 

40% 

19% 

41% 

34% 

17% 

48% 

There was more instructionally focused dialogue 
with my supervisor. 

45% 

19% 

36% 

37% 

19% 

43% 

33% 

17% 

50% 

The results of my evaluation will be used to 
inform my professional development plan for 
next year. 

50% 

30% 

20% 

43% 

31% 

26% 

39% 

31% 

31% 

Capacity Buiiding 

1 need support in... 

Gaining timely access to student data. 

44% 

14% 

42% 

49% 

14% 

37% 

51% 

18% 

31% 

Analyzing student data for action. 

44% 

12% 

44% 

48% 

13% 

39% 

48% 

17% 

35% 

Using pre/post assessments. 

36% 

12% 

52% 

39% 

13% 

48% 

40% 

16% 

44% 

Using Maryland's College and Career Ready 
Standards. 

54% 

19% 

27% 

58% 

19% 

24% 

56% 

20% 

24% 

Receiving SLO feedback from school or district 
administration. 

48% 

19% 

32% 

51% 

19% 

30% 

51% 

20% 

29% 

Using data from SLOs and observations to 
strengthen school improvement planning. 

48% 

21% 

31% 

50% 

22% 

28% 

48% 

24% 

28% 

Using data from SLOs and observations to make 
improvements in my instructional practices (for 
teachers only). 

47% 

17% 

36% 

50% 

17% 

33% 

48% 

20% 

32% 

Using data from SLOs and observations to make 
improvements in my instructional supervision (for 
principals only). 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 


Note: D = Total Disagree, a composite of strongly disagree/disagree. U = Undecided. A = Total Agree, a composite of strongly 
agree/agree. 
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Table 8. Teachers’ Responses by District Poverty Level (n=1 4,905) 



Low Poverty 
(18-30% FRL) 

Medium Poverty 
(32-49% FRL) 

High Poverty 
(52-84% FRL) 

A 

U 

D 

A 

U 

D 

A 

U 

D 

Systemic Context 

The expectations are clear for teachers and 
principals in the TPE system. 

48% 

20% 

32% 

52% 

20% 

28% 

49% 

20% 

31% 

TPE is based on principles of continuous 
improvement. 

60% 

22% 

18% 

61% 

23% 

15% 

60% 

24% 

16% 

TPE recognizes the scope of an educator’s 
roles and responsibilities. 

43% 

20% 

37% 

50% 

21% 

29% 

46% 

23% 

31% 

The State's support of the implementation of 
the TPE system is helpful. 

21% 

39% 

40% 

26% 

40% 

34% 

30% 

37% 

33% 

My district’s support of the implementation of 
the TPE system is helpful. 

39% 

31% 

30% 

42% 

30% 

27% 

38% 

31% 

31% 

My school’s support of the implementation of 
the TPE system is helpful. 

57% 

22% 

21% 

57% 

23% 

20% 

50% 

25% 

25% 

The school community’s support of the 
implementation of the TPE system is helpful. 

25% 

43% 

33% 

28% 

42% 

30% 

29% 

39% 

33% 

Quality of TPE Frameworks and Processes 

Teacher evaluation frameworks and processes... 

Respect educators’ professional knowledge and 
skills. 

51% 

17% 

32% 

57% 

16% 

27% 

53% 

18% 

29% 

Utilize validated observation 
measures/instruments. 

42% 

24% 

34% 

45% 

23% 

32% 

48% 

22% 

30% 

Are conducted by observers/evaluators 
qualified to do the evaluation. 

61% 

19% 

20% 

64% 

18% 

17% 

58% 

19% 

23% 

Provide a summative rating of educator 
performance. 

47% 

23% 

29% 

57% 

21% 

22% 

49% 

23% 

28% 

Provide useful feedback to teachers. 

46% 

26% 

29% 

53% 

23% 

24% 

50% 

23% 

27% 

Encourage reflection on instructional practices. 

65% 

17% 

19% 

67% 

17% 

16% 

67% 

17% 

16% 

Lead to improved decisions related to 
instructional approaches. 

45% 

27% 

28% 

50% 

26% 

23% 

50% 

25% 

25% 

Student Learning Objectives (SLOs) 

Teachers receive... 

Information about developing high quality SLOs. 

53% 

15% 

32% 

48% 

17% 

35% 

47% 

14% 

39% 

Information about the use of SLO components. 

53% 

17% 

30% 

48% 

18% 

34% 

46% 

16% 

38% 

Resources supporting the use of student 
baseline data. 

45% 

19% 

36% 

41% 

21% 

38% 

43% 

18% 

39% 

Guidance in selecting learning content for SLOs. 

53% 

15% 

32% 

50% 

16% 

34% 

45% 

16% 

39% 

Guidance in selecting research-based 
instructional strategies for SLOs. 

38% 

19% 

42% 

37% 

21% 

43% 

36% 

19% 

45% 

Information on connecting Maryland's College 
and Career-Ready Standards to SLOs. 

34% 

23% 

43% 

33% 

24% 

43% 

33% 

23% 

45% 

Recommendations for selecting pre/post 
assessments for SLOs. 

46% 

15% 

39% 

49% 

18% 

33% 

49% 

16% 

35% 

Valid and reliable approaches to setting growth 
targets for SLOs. 

41% 

19% 

40% 

38% 

21% 

41% 

38% 

20% 

42% 
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Low Poverty 
(18-30% FRL) 

Medium Poverty 
(32-49% FRL) 

High Poverty 
(52-84% FRL) 

A 

U 

D 

A 

U 

D 

A 

U 

D 

Professionai development that is informed by 
the SLO process. 

45% 

19% 

35% 

42% 

22% 

36% 

42% 

18% 

40% 

Guidance in developing strategies for student 
differentiation or accommodation where 
appropriate. 

38% 

19% 

43% 

38% 

21% 

41% 

36% 

20% 

44% 

This Year’s Experience 

This year, I believe... 

Data were used to develop my SLOs and to 
improve my teaching. 

57% 

17% 

26% 

51% 

18% 

30% 

56% 

17% 

27% 

There were opportunities to confer with my 
principal about my SLOs. 

75% 

8% 

17% 

73% 

10% 

18% 

60% 

11% 

29% 

There were opportunities to engage in a mid- 
interval review and refinement of my SLOs. 

72% 

9% 

19% 

60% 

12% 

28% 

48% 

15% 

37% 

Reflection on my instructional practices 
deepened. 

43% 

21% 

37% 

44% 

20% 

36% 

45% 

21% 

34% 

There was more instructionally focused dialogue 
with my colleagues. 

40% 

18% 

42% 

42% 

18% 

39% 

44% 

19% 

37% 

There was more instructionally focused dialogue 
with my supervisor. 

38% 

19% 

43% 

41% 

19% 

40% 

39% 

20% 

41% 

The results of my evaluation will be used to 
inform my professional development plan for 
next year. 

43% 

32% 

26% 

46% 

30% 

23% 

47% 

28% 

24% 

Capacity Buiiding 

I need support in... 

Gaining timely access to student data. 

45% 

16% 

39% 

47% 

15% 

37% 

52% 

13% 

35% 

Analyzing student data for action. 

44% 

14% 

43% 

47% 

14% 

39% 

50% 

13% 

37% 

Using pre/post assessments. 

36% 

14% 

51% 

38% 

14% 

48% 

43% 

13% 

44% 

Using Maryland's College and Career Ready 
Standards. 

54% 

19% 

27% 

55% 

20% 

25% 

58% 

18% 

24% 

Receiving SLO feedback from school or district 
administration. 

46% 

20% 

35% 

49% 

21% 

30% 

59% 

17% 

24% 

Using data from SLOs and observations to 
strengthen school improvement planning. 

45% 

22% 

33% 

48% 

23% 

29% 

56% 

20% 

24% 

Using data from SLOs and observations to 
make improvements in my instructional 
practices (for teachers only). 

44% 

18% 

38% 

50% 

18% 

32% 

55% 

17% 

28% 

Using data from SLOs and observations to 
make improvements in my instructional 
supervision (for principals only). 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 


Note: D = Total Disagree, a composite of strongly disagree/disagree. U = Undecided. A = Total Agree, a composite of strongly 
agree/agree. 
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